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FOREWORD 


The  Instructional  Quality  Inventory  ( I Q I )  was  developed  in  support  of 
Navy  Decision  Coordinating  Paper,  Education  and  Training  Development 
(NDCP-Z0108-PN),  under  subproject  P.30A,  Adaptive  Experimental  Approach 
to  Instructional  Design,  and  the  sponsorship  of  the  Deputy  Chief  of 
Naval  Operations,  Manpower,  Personnel,  and  Training  (0P-01).  The 
overall  objective  of  the  subproject  is  to  develop  an  empirically-based 
instructional  design  support  system  to  aid  developers  in  deciding  on 
instructional  alternatives  based  on  cost/benefits  and  specified 
resource  limitations.  The  purpose  of  the  IQI  is  to  provide  quality 
control  and/or  evaluation  procedures  for  instructional  development. 

This  report  is  the  second  in  a  series  of  four  on  the  IQI  procedures. 

It  provides  a  user's  manual  for  the  IQI  process,  and  includes  annotated 
examples  of  all  IQI  procedures.  The  other  three  reports  are: 

1.  The  Instructional  Quality  Inventory:  Volume  I,  Introduction 
and  Overview  (NPRDC  Special  Report  79-3). 

2.  The  Instructional  Quality  Inventory:  Volume  III,  Training 
Workbook,  which  will  provide  additional  examples  and  practice 
on  the  IQI  procedures  (to  be  published  in  1979). 

3.  The  Instructional  Quality  Inventory:  Volume  IV,  Job  Perfor¬ 
mance  Aid,  which  contains  brief  versions  of  the  procedures 
contained  in  this  volume.  (NPRDC  Special  Report  79-5). 

When  these  four  reports  have  been  published,  previous  training  manuals 
(NPRDC  Special  Report  77-14  and  Technical  Note  78-5)  will  have  been 
superseded. 

The  IQI  is  intended  for  use  by  the  Chief  of  Naval  Education  and  Training; 
the  Chief  of  Naval  Technical  Training;  the  Commander  Training  Command, 
Atlantic;  the  Commander  Training  Command,  Pacific;  and  all  other  Navy 
activities  concerned  with  the  development,  revision,  or  acquisition  of 
instructional  programs.  Prospective  users  of  the  IQI  are  invited  to 
contact  this  command  for  assistance  in  implementation. 


DONALD  F.  PARKER 
Commanding  Officer 
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INTRODUCTION 


The  Instructional  Quality  Inventory 

The  Instructional  Quality  Inventory  (i'QI)  is  a  set  of  procedures 
for  quality  control  of  instructional  development,  designed  to  parallel 
and  supplement  the  ISD  process.  The  IQI  procedures  can  also  be  used 
to  evaluate  existing  instruction,  and  can  be  used  as  evaluation  or 
acceptance  tools  for  instructional  programs  obtained  through  contract. 

There  are  four  volumes  in  the  IQI  series: 

1.  Volume  I  (NPRDC  Special  Report  79-3)  provides  an  introduction 
and  overview  of  the  IQI. 

2.  Volume  II  (this  volume)  includes  a  complete  description  of 
all  IQI  procedures,  and  gives  examples  of  their  use. 

3.  Volume  III  (to  be  published  in  1979)  provides  additional 
examples  and  practice  on  the  IQI  procedures. 

4.  Volume  IV  (NPRDC  Special  Report  79-5)  is  a  Job  Performance 
Aid.  It  contains  a  brief  version  of  each  IQI  procedure. 

Volume  I  should  be  read  prior  to  reading  this  volume.  Also,  each  of 
the  procedures  from  Volume  IV  is  reproduced  in  this  volume. 


About  this  Manual: 

This  manual  contains  five  chapters.  Chapter  1  contains  information 
on  classification  which  is  necessary  for  the  other  chapters.  Chapters 
2  through  5  contain  the  main  IQI  procedures.  The  chapters  are  as  follows: 

1.  Classification.  The  IQI  procedures  are  based  on  a  system  for 
classifying  objectives,  test  items,  and  instructional  components. 
Classification  is  determined  according  to  (a)  what  the  student 

is  required  to  do  with  the  information  to  be  learned,  and  (b) 
what  type  of  information  the  student  is  learning. 

2.  Objective  Adequacy.  Since  good  instruction  depends  on  careful 
s'pecifi cation  of  learning  objectives,  the  first  IQI  procedure 
is  to  assure  the  adequacy  of  objectives.  This  is  done  by 
classifying  each  objective,  and  judging  whether  or  not  it 
accurately  reflects  the  intended  student  performance  after 
training. 
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Procedure  for  Classification 


Step  1.  Determine  the  TASK  LEVEL. 

a.  Determine  whether  the  student  is  to  REMEMBER  or  USE 
information. 

b.  If  the  student  is  tc  USE  information  determine  whether 
the  task  level  is  USE-AIDED  or  USE-UNAIDED. 

Step  ?..  Determine  the  CONTENT  TYPE. 

a.  If  the  student  must  recall  or  recognize  names,  parts, 
locations,  functions,  dates,  places,  etc.,  then  the 
content  type  is  FACT. 

b.  If  the  student  must  remember  characteristics  of  similar 
objects,  events,  or  ideas,  OR  if  the  student  must  sort 
or  classify  objects,  events,  or  ideas  according  to 
characteristics,  then  the  content  type  is  CATEGORY. 

c.  If  the  student  must  remember  a  sequence  of  steps  which 
apply  to  a  single  situation,  OR  if  the  student  must 
apply  the  steps  to  a  single  piece  of  equipment  or  a 
single  situation,  then  the  content  type  is  PROCEDURE. 

d.  If  the  student  must  remember  a  sequence  of  steps  and 
decisions  which  apply  in  a  variety  of  situations,  OR  if 
the  student  must  apply  the  sequence  across  a  variety 

of  situations  or  types  of  equipment,  then  the  content 
type  is  RULE, 

e.  If  the  student  must  remember  how  or  why  things  work 
the  way  they  do,  or  cause-effect  relationships,  OR 
if  the  student  must  use  his  knowledge  to  explain  how 
things  work,  or  predict  effects  from  causes,  then  the 
content  type  is  PRINCIPLE. 


This  procedure  is  explained  in  more  detail  on  the  following  pages. 
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Explanation  for  STEP  1,  Determine  Task  Level. 


The  first  step  in  the  classification  procedure  involves  deciding 
whether  the  student  is  required  to  remember  inf  oration,  or  whether 
the  student  .rust  use  information  to  perform  some  task .  The  remember- 
use  distinction  is  a  simple  one.  The  determination  can  usually  be  made 
by  looking  at  the  ACTION  in  the  objective  or  test  item.  Typical  action 
verbs  are  listed  below.  The  ones  on  the  left  usually  indicate  'remember 
to.sks,  while  the  ones  on  the  right  usually  indicate  use-level  tasks. 


REMEMBER 

USE 

name 

apply 

operate 

state  (from  memory) 

classify 

repair 

li-t  (from  memory) 

analyze 

adjust 

recall 

derive 

alibrate 

remember 

demonstrate 

remove 

relate 

discriminate 

replace 

write  (from  memory) 

evaluate 

assemble 

recognize 

solve 

disassemble 

explain  (from  memory) 

prove 

calculate 

describe  (from  memory) 

sort 

explain  (on  the 
maintain 

basis  of  other 

compute 

troubleshoot 

predict 

load 

perform 

determine 

• 

unload 

.  ( there  are  many  other  USE  a< 

If  thr  task  level  is  USE,  the  next  step  is  to  determine  whether 
an  AID  is  given.  This  can  be  done  by  looking  at  the  "conditions"  part 
of  the  objective  or  test  or  practice  item.  Anything  that  replaces  the 
need  for  MEMORY  counts  as  an  AID. 

ADDS  include:  a  list  of  procedure  steps  from  a  tech. 

manual  or  MRC  card , 

a  formula  for  solving  problems 

a  list  or  table  or  chart  of  characteristics 

a  statement  of  a  principle. 


Normal  tools,  materials ,  etc.,  are  NOT  aids. 


Explanation  for  STEF  2,  Determine  Content  Type. 

STEP  2a:  FACTS.  Facts  are  what  you  think  they  are.  They  are  simple 

associations  between  objects,  events ,  names,  parts,  functions, 
loco/ ions,  dates,  etc.  Facts  don't  have  to  come  in  pairs ;  there 
may  be  three  or  four  or  more  pieces  of  information  that  go 
together.  For  example,  a  student  might  have  to  remember  the 
name,  location,  and  function  of  each  of  the  parts  in  some  piece 
of  equipment . 

Key  words  or  phrases  that  may  help  identify  FACT-level  objectives 
or  test  items  are  listed  below: 

The  student  will  give  the  symbol  for  each  .... 

match  each  ...  with  its  .... 
list  the  names  of  each  .... 
recall  the  dates  of  .... 

recall  the  location  and  function  of  each  .... 
give  the  . . .  associated  with  each  . . . 

STEP  2b:  CATEGORIES.  Categories  refer  to  groups  of  similar  objects, 

events,  or  ideas .  They  are  similar,  or  are  grouped  together, 
because  they  have  characteristics  in  common.  Category  tasks 
nearly  always  involve  classification  or  sorting  cn  the  basis  of 
these  critical  characteris  ;ics.  At  the  REMEMBER  level  for 
categories,  the  student  is  required  to  remember  these  charac¬ 
teristics,  and  hew  they  go  together.  At  the  USE  level ,  the 
student  is  required  to  identify ,  sort ,  or  classify  things 
according  to  these  characteristics. 

Key  words  or  phrases  that  may  help  identify  CATEGORY-level 
objectives  or  Ust  items  are  listed  below: 


REMEMBER  LEVEL: 
The  student  will 


recall 

name  the 

describe 

give 


characteristics 

features 

definition  of  each 
attributes 


type  of  ... 
kind  of  ... 
category  of  .. . 
classification 
situation 


USE  LEVEL: 

The  student  will 


sort 

classify 

categorize  each  . . .  according  to 

identify 

recognize 

choose 

select 


type 

kind 

characteristics 

definition. 

features 
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STEP  2a:  PROCEDURES.  A  procedure  is  a  sequence  of  steps  that  must  be 

performed  in  order.  Procedures  are  always  applied  in  the  same 
way,  on  situations  or  equipment  that  do  not  change.  At  the 
REl-EMBER- levs l  for  procedures ,  the  student  is  required  to 
remember  the  steps  and  their  order.  .4t  the  USE-fevel ,  the 
student  is  to  perform  the  procedure. 


Key  words  or  phrases: 


REMEMBER  LEVEL:  reca 1 7 

list 

The  student  will  name  the 
state 
give 


stems 

process 

procedure  for 
sequence 


operating 
performing 
mai ntaining 
lighting  off 
etc. 


USE  LEVEL: 


The  student  will 


apply 

operate 

repair 

adjust 

calibrate 

eta. 


remove 

replace 

assemble 

produce 

destroy 


STEP  2d:  RULES.  A  rule,  like  a  procedure,  is  a  sequence  of  steps. 

However,  rules  can  be  applied  in  a  variety  of  situations  or 
on  a  variety  of  different  equipments.  Because  they  apply 
in  a  variety  of  situations,  rules  sometimes  have  complicated 
decision  steps. 


Formulas  and  mathematical  calculations  always  involve  the 
use  of  rules,  unless  the  student  has  a  calculator  or  computer 
that  does  it  for  him. 

Key  words  or  phrases: 


REMEMBER  LEVEL: 

recall 

formula 

solving 

name 

rule 

deriving 

The  student  will 

state  the 

laxo  for 

proving 

give 

process 

calculating 

remember 

steps 

determining 

etc. 


USE  LEVEL: 

The  student  will 


solve 

derive 

prove 

calculate 

determine 

etc. 


find 

translate 

program 

add 

subtract 
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STEP  2e:  PRINCIPLES.  Principles  involve  explanations  of  why/how  things  work 
the  way  they  do,  or  predictions  about  "what  would  happen  if  . . . . " 
Principles  are  based  on  cause-effect  relationships ,  theoretical 
statements,  statistical  associations,  or  physical  or  scientific 
"laws."  At  the  REMEMBER- level,  the  student  must  recall  reasons, 
causes  and  effects,  theoretical  statements,  etc.  At  the  USE-level, 
the  student  must  use  his  knowledge  to  give  a  prediction  or 
explanation  about  how  something  works,  or  what  is  likely  to  happen, 
or  why  something  isn't  working  the  way  it  should. 

Key  words  or  phrases: 


REMEMBER  LEVEL: 
The  student  will 


USE  LEVEL: 

The  student  will 


recall 

the 

remember 

the 

state 

how 

describe 

why 

discuss 

the 

analyze 

evaluate 

explain 

diagnose 

troubleshoot 

predict 

principle  of 
explanation  of 


reasons  for 


NOTE:  It  is  sometimes  difficult  to  determine  whether  a 
PRINCIPLE  objective  is  REMEM3ER  or  USE,  Look  at 
the  following  example  objective: 

"The  student  will  explain  the  operation  of  a 
rotary- gear  pump. " 

This  objective  could  be  either  USE  or  REMEbJBER. 

It  would  be  USE  if  the  student  had  not  been 
taught  about  rotary-gear  pumps  and  was  required 
to  use  his  knowledge  of  hydraulic  principles 
to  explain  their  operation.  If  the  instruction 
had  dealt  with  rotary  gear  pumps  specifically, 
then  the  objective  would  be  REMEMBER. 

If  you  encounter  an  objective  of  this  type,  you 
must  determine  if  the  information  has  been  taught 
(if  you  are  evaluating  an  existing  course)  or 
if  the  information  is  to  be  taught  ( if  you  ai j 
evaluating  the  objectives  of  a  course  under 
development) .  If  the  information  is  or  will  be 
included  in  the  instruction, then  the  task  level 
is  REMEMBER.  If  the  student  is  required  to  use 
principles  to  explain  things  he  has  not  been  taught 
specifically,  then  the  task  level  is  USE. 

Actually,  it  is  best  to  make  the  objective  precise 
in  the  first  place.  For  example: 

"The  student  will  recall  the  explanation  of  the 
operation  of  a  rotary-gear  pump.  " 
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Additional  Explanation  for  STEF  2 ,  Determine  Content  Type. 


On  the  preceding  pages,  we  have  given  definitions  and  key  words  for 
each  content  type ;  however,  determining  content  type  can  still  be 
difficult.  In  the  following  sections,  we  will  give  a  schematic  representation 
of  each  content  type ,  and  further  guidelines  for  distinguishing  among 
content  types. 


FACTS 

Facts  are  simple  associations  between  names,  objects,  etc.  The 
task  is  always  for  the  student  to  recall  them  or,  given  one  pact  of 
the  fact,  to  recall  the  other  parts. 


fact  1:  0 — 0 — 0 — 0 

fact  2:  0 — 0 — 0- — 0 


fact  n:  0 — 0—0 — 0 


Example: 

"The  student  will  recall  in  writing  the  name,  location,  and  function 
of  each  dial  on  the  front  panel  of 

Dial  1:  name — location — function 

Dial  2:  name — ■ location — function 


Dial  n:  name — location — function 
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CATEGORIES 


Category  tasks  involve  sorting  cr  classifying  objects  or  events 
according  to  their  characteristics  or  features.  At  the  REMEMRFR  level, 
tne  student  must  remember  the  characteristics  and  horn  they  go  together, 
nt  the  USE  level ,  the  student  must  identify ,  sort ,  or  classify  things 
according  to  these  characteristics . 


small 

number 

of 

possible 

categories 


Example: 


Given  a  series  of  sonar  scope  display s, 
them  according  to  type  of  target. " 


the  student  will  classify 


small 

number 

of 

possible 

types 


cor  categories  at  the  REMEMBER  LEVEL ,  the  student  must  remember  the 

characteristics.  (The  middle  box  above.) 
at  the  USE  LEVEL,  the  student  gets  " inputs , "  evaluates  the 
characteristics,  and  determines  the  appropriate 
category  or  type. 
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PROCEDURES 


A  procedure  is  a  sequence  of  steps ,  performed  in  order,  on  a  single 
piece  of  equipment  or  in  a  single  situation .  At  the  REMEMBER  level ,  the 
student  must  remember  the  steps  in  order.  At  the  USE  level ,  the  student 
is  given  a  piece  of  equipment  or  a  situation,  and  must  perform  the  steps. 


Example: 

" The  student  will  field-strip  an  M-16  rifle.  " 


STEPS  OF 
PROCEDURE  FOR 
FIELD-STRIPPING 
AN  M-16  RIFLE 

[  (M-16  rifle) 

(field-stripped  rifle) 

RULES 

A  rule,  like  a  procedure,  is  a  sequence  of  steps  and  decisions. 
However,  rules  can  be  applied  in  a  variety  of  situations  or  on  a 
variety  of  equipments At  the  REMEMBER  level,  the  student  must  remember 
the  steps  and  decisions.  At  the  USE  level,  the  student  is  given  problem 
situations,  and  must  apply  the  steps  of  the  rule  to  solve  the  problem  or 
come  up  with  the  answer. 


Example:  \ 


"Given  any  two  values  of  current,  voltage,  or  resistance  in  a  circuit , 
the  student  will  use  Ohm's  Law  to  solve  for  the  third  value," 


PRINCIPLES 


Principles  involve  explanations  of  why  or  how  things  work  the 
way  they  do3  or  predictions  about  "what  would  happen  if  ...." 

At  the  REMEMBER  level 3  the  student  must  recall  reasons 3  causes  and 
effects 3  theoretical  statements 3  etc.  At  the  USE  lev  el 3  the  student 
uses  his  knowledge  of  causes  and  effects 3  or  theories 3  to  explain  or 
predict. 


Example: 


"Based  on  his  knowledge  of  electronic  theory 3  the  student  will 
predict  the  effect  in  the  circuit  shown  below  if  the  load 
resistance3  or  the  filter  capacitor 3  were  shorted. " 


Note:  The  reason  that  principles  are  taught  is  that 

they  apply  in  a  variety  of  situations.  They  allow 

the  student  to  make  a  variety  of  "what  would  happen  if. . . " 

predictions. 

However 3  if  the  student  is  only  -required  to  remember 
one  cause  and  one  effect3  then  it  should  le  classified  as  a 
fact .  For  example 3  if  a  particular  symptom  in  a  piece 
of  electronic  equipment  always  means  that  a  particular 
part  is  damaged3  that's  a  fact. 


For  Categories 3  Procedures 3  Rules ,  and  Principles  at  the  REMEMBER 
level  the  student  has  to  remember  whatever  is  in  the  middle  box  on 
the  diagrams  above.  At  the  USE  level 3  the  student  has  to  perform 
the  whole  task. 
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Further  Guidelines  for  Classification. 


Remember  the  Job.  The  most  important  thing  to  remember  when 
attempting  to  classify  objectives  is  the  job.  The  classification 
scheme  was  designed  so  that  classification  depends  on  the  job 
requirements.  The  most  important  requirement  to  consider  is 
whether  or  not  the  student  will  have  to  deal  with  objects  or 
situations  he  has  not  seen  or  encountered  during  training.  For 
the  FACT  and  PROCEDURE  content  types ,  this  does  not  occur.  Facts 
by  definition  must  be  presented  during  training.  The  job 
requirements  for  procedures  involve  single  pieces  of  equipment 
or  single  situations,  and  the  student  does  not  have  to  "generalize" 
to  new  equipments  or  situations.  In  other  words,  everything  the 
student  needs  to  know  is  presented  during  training. 

On  the  other  hand,  there  are  some  job  situations  that  require 
the  student  to  deal  with  so  many  possible  objects ,  events,  ideas , 
problems ,  or  situations,  that  it  would  be  impossible  to  include 
all  of  them  during  training.  In  this  case,  the  training  program 
is  designed  so  that  the  student  will  be  able  to  deal  with  new 
cases.  CATEGORIES,  RULES,  and  PRINCIPLES  are  used  in  the  clas¬ 
sification  scheme  to  cover  this  situation. 

‘The  CATEGORY  content  type  is  used  when  the  job  requires  that 
a  large  number  of  possible  objects,  events,  etc.  be  classified  into, 
or  identified  as  a  member  of,  one  of  a  small  number  of  particular 
categories.  Instead  of  having  to  remember  each  object  and  its 
classification,  the.  student  is  given  characteristics  for  each 
category,  which  allow  him  to  classify  objects,  etc.,  he  has  not 
seen  before. 

The  RULE  content  type  is  used  when  the  job  requires  that 
a  large  number  of  problems  be  solved  or  that  a  complicated 
sequence  of  steps  be  performed  on  a  large  number  of  different 
objects,  events,  etc.  Instead  of  having  to  remember  each 
problem  or  go  through  the  steps  on  each  object, the  student 
is  taught  a  RULE  which  allows  him  to  deal  with  problems, 
objects,  and  events  he  has  not  seen  before. 

The  PRINCIPLE  content  type  is  used  when  the  job  requires 
prediction  or  interpretaticn  of  a  large  number  of  possible 
situations,  events,  effects,  etc.  Instead  of  having  to  remember 
each  possible  situation  or  event  and  its  effects,  the  student  is 
given  a  PRINCIPLE  which  summarizes  the  "how"  or  "why"  of  general 
situations  or  which  allows  the  student  to  predict  what  is  likely 
to  occur  in  a  variety  of  situations. 
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Problems  in  Classification.  Sometimes  classification  can  be 
tricky.  There  can  be  confusion  between  FACTS  and  CATEGORIES,  and 
between  PROCEDURES  and  RULES.  The  way  to  resolve  problems  is  to 
"REMEMBER  THE  JOB";  that  is  to  consider  carefully  what  the  student 
must  be  able  to  do  after  instruction . 

Again,  the  most  important  thing  to  consider  is  whether  the 
student  will  have  to  deal  wi.th  objects  or  situations  that  he  has 
riot  seen  during  training.  For  example,  if  the  student  were  required 
to  sort  or  classify  things  according  to  their  characteristics,  and 
if  the  student  on  the  job  were  going  to  be  dealing  with  things  not 
seen  during  training,  then  the  objective  would  be  a  CATEGORY. 

However,  suppose  instead  that  there  were  only  seven  objects  the 
student  would  ever  see.  Then,  it  would  be  more  efficient  to  teach 
each  object  and  its  category  name  as  a  fact  (seven  facts  total). 

Similarly,  RULES  are  taught  so  that  the  student  can  apply  his 
knowledge  to  situations  he  won't  have  seen  in  training.  However, 
suppose  the  situations  are  so  similar  that  "if  you've  seen  one , 
you ’ve  seen  them  all. "  This  would  be  more  efficiently  taught  as 
a  PROCEDURE. 

On  the  other  hand,  some  tasks  look  at  first  like  PROCEDURES , 
but  turn  out  to  be  more  complicated.  An  expert  who  really  knows 
the  job  can  help  you  make  the  decision. 

Example:  FACT  vs.  CATEGORY 

"Given  a  variety  of  metal  fasteners,  the  student  will  sort  them 
according  to  type  (bolts,  screws ,  studs,  or  rivets) . " 

This  could  be  taught  as  a  CATEGORY:  the  student  could  be  taught 
the  characteristics  of  bolts  (fine  threads,  blunt  end ,  etc.), 
the  characteristics  of  screws  (coarse  threads,  pointed  end,  etc.), 
the  characteristics  of  studs  (no  head,  fine  threads,  etc.),  and 
the  characteristics  of  rivets  (no  threads,  etc.).  However ,  one 
bolt  is  pretty  much  the  same  as  any  other  bolt,  and  the  same 
for  screws,  studs,  and  rivets,  except  that  they  come  in  different 
sizes.  Therefore,  it  might  be  more  efficient  to  teach  these 
as  four  FACTS:  bolt  -  appearance,  screw  -  appearance,  etc. 

The  confusion  here  can  be  solved  if  the  job  requirements  are 
determined,  if  there  are  lots  of  different  metal  fasteners,  and 
the  student  will  see  new  bolts,  etc.,  on  the  job,  then  the 
content  type  is  CATEGORY.  If  there  are  only  a  few,  and  they  're 
all  nearly  alike,  then  the  content  type  is  FACT. 

Example:  PROCEDURE  vs.  RULE 

"Given  a  word  in  print,  correctly  spelled,  the  student  will  look 
up  the  word  in  a  dictionary,  and  state  its  definition  orally. " 

This  might  appear  to  be  a  RULE:  There  are  a  large  number  of 
possible  words  (inputs),  and  a  laxge  number  of  possible  definitions 
(outputs).  However,  since  the  spelling  is  given,  its  easy  to  look 
up  the  word:  Find  the  first  letter  of  the  word,  find  that  chapter 
in  the  dictionary,  find  the  second  letter,  find  that  section  of 
the  chapter,  etc.  This  is  most  efficiently  taught  as  a  PROCEDURE. 

However,  suppose  the  word  was  given  orally  and  not  spelled.  This 
would  then  be  a  fairly  complicated  RULE,  involving  listening  skills, 
phonemic  translations,  etc. 


Why  do  we  need  a  classification  scheme  at  all? 


The  classification  scheme  is  essential  for  two  reasons;  First,  it 
makes  consistency  judgments  between  objectives ,  test  items,  cmd 
instructional  presentations  possible.  If  we  didn't  classify  objectives 
and  test  items,  all  we  could  say  is  "This  is  an  objective  and  this  is 
a  test  item,  cmd  they  don't  look  too  different."  The  classification 
scheme  allows  us  to  be  more  precise  about  what  objectives,  test  items, 
and  instruction  are  asking  the  student  to  do  or  learn. 

Second,  the  classification  scheme  makes  adequacy  judgments  about 
objectives,  test  items,  and  instructional  presentations  possible. 

In  fact,  the  classification  scheme  was  designed  so  that  the  classification 
of  an  objective  has  implications  for  the  way  the  instruction  for  that 
objective  should  look.  For  example,  the  instruction  for  a  Use-unaided 
Rule  should  be  different  than  the  instruction  for  a  Remember- Category 
objective.  The  most  important  differences  occur  between  the  Remeniber 
and  Use  task  levels,  and  between  the  content  types  that  do  not  require 
generalisation  (Facts  and  Procedures) ,  and  those  that  do  (Categories, 
Rules,  and  Principles) .  When  generalization  is  required,  there  will 
be  more  examples  and  practice  items  covering  a  wider  range  of  difficulty. 


In  the  rest  of  this  chapter ,  several  example  objectives  are  given, 
together  with  their  classifications ,  and  some  explanation  about  each 
classification.  In  Volume  III,  Chapter  1,  there  are  sample  objectives 
you  can  classify  yourself  for  practice. 
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EXAMPLES 


In  this  section,  sample  objectives  and  sample  test  items  are 
given.  They  are  classified  according  to  the  scheme  presented  in  the 
first  part  of  this  chapter,  and  explanation  about  the  classification 
is  given. 

Some  of  the  objectives  and  test  items  given  below  will  be 
difficult  to  classify.  There  are  three  reasons  for  this.  First, 
many  of  them  are  not  "good"  ones;  they  are  written  in  such  a  way 
that  it  may  not  be  clear  what  behavior  is  required  or  what  content 
is  to  be  taught.  (They  are,  however,  fairly  typical.)  Second,  all 
of  them  are  taken  out  of  context  and  may  deal  with  unfamiliar  topics. 
Therefore,  they  are  difficult  to  classify,  because  information  about 
the  job  is  not  provided.  Third,  some  examples  were  chosen  deliberately 
to  be  hard  to  classify,  so  that  classification  problems  could  be 
illustrated. 

For  the  reasons  given  above,  the  reader  should  not  expect  to 
be  able  to  classify  perfectly  these  objectives  and  test  items  (or 
any  others)  immediately.  In  fact,  it  is  better  not  to  attempt 
immediate  classification.  Instead,  since  the  most  Important  step 
in  classification  is  REMEMBER  THE  JOB,  the  reader  should  learn  to 
ask  the  "right"  questions  of  job  experts,  so  that  bad  objectives  can 
be  revised,  and  so  that  unfamiliar  topic  areas  can  be  classified 
reliably.  The  examples  are  intended  to  illustrate  this  "question 
asking"  process. 

In  this  section,  we  are  concerned  only  with  the  classification 
of  each  example.  In  the  next  chapter,  we  will  deal  with  the 
adequacy  of  objectives. 
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1.  OBJECTIVE: 
Task  Level? 

Content  Type? 


2.  OBJECTIVE: 

Task  Level? 

Content  Type? 


"The  student  will  state  the  rule  for  finding  total 
inductance  in  a  series  circuit." 

REMEMBER 


Does  the  student  have  to  remember  something ,  or 
\ perform  a  task?  In  this  case,  the  student  simply 
has  to  recall  the  correct  rule  for  finding  total 
inductance.  Therefore ,  the  task  level  is  Remember. 

RULE 


The  student  must  remenber  a  rule  for  SOLVING  for 
total  inductance.  "SOLVE"  is  a  keyword  for  RULE. 
The  formula  for  total  inductance  involves  a  series 
of  mathematical  operations  applicable  in  any  series 
circuit  having  inductors  with  various  possible 
values.  The  process ,  then,  is  a  series  of  steps 
which  apply  across  situations. 

It  is  possible  to  confuse  a  piece  of  information 
taught  at  the  Remember  level  for  Categories, 
Procedures ,  Rules ,  or  Principles,  with  Remember- 
Fact  information.  The  difference  is  that  Facts 
cannot  be  used  immediately.  In  this  case,  the 
student  could  use  the  rule  to  solve  inductance 
problems. 


"Given  pictures  of  the  collar  devices  for  different 
ranks  of  Naval  officers,  the  student  will  identify 
the  ranks  they  represent." 

REMEMBER 


The  task  here  is  "I  give  you  a  picture  of  a  collar 
device ,  and  you  tell  me  the  rank. "  Because  there 
is  a  linri-ted  number  of  different  collar  devices, 
the  student  can  learn  each  device  and  its  name. 

The  task  is  Remember. 

FACT 


The  content  type  is  Fact,  because  the  student  has 
to  memorize  pairs  of  symbols  and  names. 

Suppose  instead  that  there  were  a  large  number  of 
different  ranks  and  collar  devices,  but  that 
different  groups  of  them  had  similar  characteristics . 
If  the  student’s  task  were  to  look  at  a  collar 
device  and  identify  which  group  it  belonged  to, 
then  this  objective  would  be  Use-unaided  Category. 


3.  OBJECTIVE:  "Given  any  resistor  with  four  color  bands,  the  student 

will  state  the  ohmic  value  indicated  by  the  color 
bands." 

Task  Level?  USE-UNAIDED 


If  there  were  only  a  limited  number  of  resistors 
the  student  would  ever  have  to  deal  with ,  he  could 
memorize  each  resistor's  colors  and  their  values , 
and  the  task  level  would  be  Remember.  However , 
there  are  many  different  resistors  with  many 
different  color  patterns.  Therefore,  the  student 
must  be  given  a  scheme  for  determining  ohmic  value 
from  the  color  pattern.  When  the  student  applies 
this  scheme  to  any  particular  resistor,  he  is 
USING  the  scheme.  If  he  has  no  memory  aid,  then 
the  task  level  is  Use-Unaided. 

Content  Type?  PROCEDURE _ 

In  order  to  make  the  content  type  decision ,  we 
need  to  know  more  about  the  content  than  is  given 
in  the  objective.  In  particular ,  we  need  to  know 
what  the  scheme  for  determining  ohmic  value  ‘from 
the  colors  is.  This  is  a  good  time  to  consult  a 
subject-matter  expert.  It  turns  out  that  the 
scheme  is  a  fairly  simple  sequence  of  steps: 
the  first  two  color  bands  indicate  the  first  two 
significant  figures  of  the  ohmic  value;  the  third 
band  is  the  number  of  zeros  to  add.  (The  fourth 
band  determines  tolerance ,  not  value. )  If  the 
scheme  were  more  complicated,  and  involved  complex 
calculations,  the  content  type  would  be  Rule. 

This  one  is  simple  enough  to  be  a  Procedure. 

This  objective  should  not  be  confused  with  simply 
learning  the  meanings  of  the  colors.  Those  are 
Facts  which  the  student  must  remember.  Those 
facts  support  this  Use-level  objective. 


This  objective  is  a  good  example  of  why  it  is  a  good  idea  not 
to  attempt  immediate  classification.  More  detail  about  the  job  the 
student  will  perform  is  needed.  For  example,  the  word  "state"  in  the 
objective  is  misleading;  the  student  really  has  to  use  a  procedure 
to  determine  the  value  before  he  can  state  it.  "Determine"  or  "cal¬ 
culate"  would  be  a  better  action  verb,  but  that's  the  subject  of  the 
next  chapter. 
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4.  TEST  ITEM: 

Task  Level? 

Content  Type? 

5.  OBJECTIVE: 
Task  Level? 

Content  Type? 

6.  TEST  ITEM: 

Task  Level? 

Content  Type: 


"What  are  the  steps  involved  in  message  reception 
and  duplicate  checking,  as  listed  in  the  current 
edition  of  NTP-4?" 

REMEMBER 


The  student  is  asked  to  recall  the  steps ,  not  to 
do  anything  with  them. 

PROCEDURE 


"Steps"  is  a  keyword  for  Procedure,  and  in  this 
case  the  task  described  in  the  test  item  appears 
to  be  procedural.  It  would  be  a  good  idea ;  however , 
to  check  with  a  subject  matter  expert  to  make  sure. 


"The  student  will  describe  the  principles  of  operation 
of  a  rotary  gear  pump." 

REMEMBER  or  USE-UNAIDED 


It  is  difficult  to  determine  whether  this  objective 
is  Remember  or  Use-unaided  without  further  information. 
If  the  objective  had  said  "The  student  will  recall 
the  principles  . . . ,  "  then  it  would  clearly  be 
Remember.  On  the  other  hand,  if  the  objective 
had  said  "The  student  will  use  his  knowledge  of 
hydraulic  theory  to  explain  how  a  rotary  gear 
pump  works, "  then  it  would  be  Use-unaided. 

It  is  necessary  to  determine  w}iat  the  student 
must  do  after  training,  and  write  the  objective 
appropriately. 

PRINCIPLE 


This  objective  requires  the  student  to  explain  how 
or  why  the  pump  works.  The  content  type  is 
Principle. 


"Disassemble -and  reassemble  the  globe  valve  using 
your  MRC  job  program  card.  The  valve,  rags,  prussian 
blue,  gasket  material,  packing,  and  tools  are  on  the 
workbench." 

USE-AIDED 


The  MRC  job  program  card  givest  the  steps  to  be 
followed,  so  the  student  does  not  have  to  remember 
them. 

PROCEDURE 


This  is  a  series  of  steps  performed  on  a  single 
piece  of  equipment. 
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7.  TEST  ITEM: 

Task  Level? 


Content  Type? 

3.  OBJECTIVE: 
Task  Level? 

Content  Type? 


"For  each  of  the  call  signs  listed  below 
identify  whether  it  is  a  Navy  ship  call  sign, 
a  Navy  shore  call  sign,  an  indefinite  call  sign, 
a  voice  call  sign,  a  task  organization  call  sign, 
or  not  a  valid  Navy  call  sign." 

USE-UNAIDED 


The  task  level  of  this  test  item  depends  on 
complexity  of  the  task.  If  there  were  just 
a  few  call  signs,  then  it  would  be  easy  to 
memom'.ze  their  types ,  and  the  task  would  be 
Remember.  Here,  though ,  it  turns  out  that 
there  are  several  thousand  call  signs,  and 
the  student  must  use  some  classification 
scheme i  to  identify  their  type.  This  test 
item  requires  the  student  to  use  the  scheme 
with  no  memory  aid. 

CATEGORY 


In  this  case  there  is  a  large  number  of  call 
signs  which  can  be  classified  into  a  small 
number  of  categories  or  types. 


"Given  the  formula  for  Ohm's  Law,  and  two  values, 
the  student  will  solve  for  the  third  value." 

USE-AIDED 


The  student  must  use  Ohm's  Law  to  solve  for  a 
value.  In  this  case,  he  is  given  the  formula 
as  a  memory  aid.  He  does  not  have  to  remember 
it.  The  task  level  is  Use-aided. 

RULE 


"Solve"  is  a  keyword  for  Rules.  When  the  student 
is  asked  to  solve  something,  he  must  perform  a 
series  of  mathematical  operations  which  result 
in  unique  solutions  depending  on  the  values  used. 
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9.  OBJECTIVE: 


Task  Level? 


"The  student  will  recall  the  duties  of  a  gun 
captain." 

REMEMBER 


"Recall"  is  a  keyword  for  the  Remember  level. 
The  student  does  not  have  to  perform  the 
duties 3  only  remember  what  they  are. 

Content  Type?  FACT  or  PROCEDURE 


The  content  type  is  Procedure  if  the  student 
is  expected  to  remember  a  specific  sequence  of 
steps  which  the  gun  captain  must  perform. 
However ,  if  the  student  is  to  recall  only 
the  general  functions  of  a  gun  captain,  the 
objective  would  be  a  Fact.  Again,  it  is 
necessary  to  consult  a  subject-matter  expert 
to  make  this  decision. 


10.  OBJECTIVE:  "In  a  classroom  exercise,  the  student  will  file 

thirty  drill  messages  representing  Top  Secret, 
Secret,  special  category,  readdressed  general 
messages,  and  other  classified  and  unclassified 
messages  in  the  correct  files." 


Task  Level?  USE-UNAIDED 


The  student  must  perform  the  task,  with  no 
memory  aid.  The  task  level  is  Use-unaided. 

Content  Type?  CATEGORY  OR  PROCEDURE 


If  the  student  must  inspect  each  message  and 
determine  category  membership  according  to 
its  critical  characteristics,  then  the  content 
type  is  Category.  If,  however,  the  classification 
of  the  message  is  obvious  (e.g.  the  classification 
is  printed  on  the  top  line  of  the  message),  then 
this  is  a  procedural  task  involving  filing.  Once 
again,  the  services  of  a  subject-matter  expert  are 
required. 


11.  TEST  ITEM: 
Task  Level? 


"Give  the  names  of  the  components  represented  by 
the  following  schematic  symbols..." 

REMEMBER 


In  order  to  answer  this  test  item,  the  student 
would  have  had  to  memorize  the  names  of 
the  schematic  symbols;  therefore ,  the  task 
level  is  Remember. 


Content  Type?  FACT _ 

The  key  phrase  in  this  test  item  is  "Give  the 
names."  Simple  associations  between  objects 
or  symbols  and  their  names  are  Facts. 

12.  OBJECTIVE:  "Given  the  explanation  of  the  principle  of  supply 

of  a  military  fe>  ;e  from  the  text,  the  student 
will  discuss  hew  this  principle  applies  in 
Hitler's  a tack  on  Russia,  the  Battle  of  Midway, 
the  Battle  of  the  Bulge,  and  Sherman's  march 
through  Georgia." 


Task  Level?  USE-AIDED 


The  student  is  given  an  explanation  of  the 
principle ;  therefore  the  task  is  Aided. 

The  task  is  Use  because  the  student  must 
apply  the  principle  to  explain  specific 
battles.  It  is  assumed  that  these  battles  are 
not  discussed  in  the  text  and  have  not  been 
described  in  class.  If  they  had  been,  the  task 
level  would  be  Remember  and  there  would  be 
-no  reason  to  give  the  student  an  Aid. 

Content  Type?  PRINCIPLE _ 


The  student  is  given  a  principle  and  asked  to 
interpret  '‘'he  outcome  of  specific  battles. 

The  key  words  here  are  "principle"  and 
"discuss. " 
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13.  OBJECTIVE: 


"The  student  will  recall  in  writing  the  lofargram 
characteristics  for  the  follovnng  types  of  contacts: 
merchant  ship,  aircraft  carrier,  destroyer,  whale 
diesel  submarine,  nuclear  submarine." 

Task  Level?  REMEMBER 


The  student  does  not  have  to  do  anything  with 
the  characteristics  except  recall  them ;  therefore , 
the  task  level  in  Remember. 


Content  Type?  CATEGORY  or  FACT _ 

The  student  is  asked  to  memorize  characteristics  of 
several  categories  of  contacts  presumably  because 
he/she  will  later  have  to  look  at  lofargrams  and 
determine  what  the  contact  type  is.  Therefore 3 
the  content  type  is  either  Fact  or  Category 
depending  on  whether  or  not  this  identification 
requires  generalization.  If  the  lofargrams 
within  each  category  are  pretty  much  identical , 
then  the  content  tyre  would  be  Fact.  If  not3  then 
the  content  type  is  Category .  The  guidance  of 
a  subject-matter  expert  is  required. 

14.  OBJECTIVE:  "The  student  will  solve  for  inductive  reactance  in 

a  circuit,  given  frequency  and  inductance." 


Task  Level?  USE-UNAIDED 


The  task  level  is  Use  because  the  student  is 
required  to  solve  problems.  It  is  not;  aided 
because  he/she  is  not  given  the  formula. 

Content  Type?  RULE _ 


The  word  "solve"  is  a  keyword  for  Rule.  Here 
the  student  must  remember  the  formula  for 
inductive  reactance  and.  then  use  it  to  solve 
problems. 
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15.  OBJECTIVE: 


"The  student  will  perform  the  steps  required  to 
accept,  verify,  and  log  messages  to  be  transmitted 
via  teletype  tape." 

Task  Level?  USE-UNAIDED 

The  student  must  "perform  the  steps,  "  so  the 
task  level  is  Use.  The  student  has  no  memory 
aid,  so  the  task  is  Unaided. 

Content  Type?  PROCEDURE _ 

The  word  "steps"  is  a  keyword  for  Procedure  or 
Rule.  If  accepting,  verifying,  and  logging  is 
complicated  enough  so  that  doing  one  or  two  does 
not  mean  that  the  student  could  do  any  new 
message,  then  the  content  type  would  he  Rule. 

In  this  case,  though,  the  steps  are  the  same 
for  every  message,  so  the  content  type  is 
Procedure. 


16.  OBJECTIVE:  "The  student  will  state  the  principles  of  operation 

of  a  jet  pump  as  described  in  Propulsion  Engineering , 

Mod  6,  Lesson  2." 

Task  Level?  REMEMBER _ 

The  student  simply  has  to  recall  how  a  jet  pump 
works,  as  described  in  his  training  manual. 

Content  Type?  PRINCIPLE _ 

The  student  is  asked  to  remember  "how"  something 
works.  Explanations  of  how  things  work  are 
Principles. 

17.  OBJECTIVE:  "The  student  will  use  the  principle  of  electro¬ 

magnetic  induction  to  describe  the  operation  of 
an  AC  generator." 

Task  Level?  USE-UNAIDED  or  REMEMBER 

The  task  level  depends  on  what  the  student  will 
be  taughv  during  the  course ;  that  is,  on  what  the 
other  objectives  are.  If  the  student  must  use 
his  knowledge  of  electromagnetic  induction  to 
describe  something  not  yet  • caught ,  then  the  task 
is  Use-unaided.  If  the  student  had  been  taught 
the  principles  of  operation  of  an  AC  generator 
on  the  basis  of  electromagnetic  induction,  then 
all  the  student  would  have  to  do  is  Remember. 

Content  Type?  PRINCIPLE 

The  student  must  explain  "how"  an  AC  generator  works. 
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18.  OBJECTIVE:  "Given  the  guidelines  for  determining  Message 

security  classification,  the  student  will  determine 
the  security  classification  (Top  Secret,  Secret, 
Confidential,  or  Unclassified)  for  outgoing 
messages." 

Task  Level?  USE-AIDED 


The  student  must  'perform  a  task:  determine 
appropriate  security  classifications  for  a 
variety  of  messages.  The  task  level  is  Use. 

To  the  extent  tliat  the  given  guidelines  provide 
sufficient  information  to  determine  classifications 
for  varied  messages,  the  task  is  Aided. 

Content  Type?  .  CATEGORY _ 


It  is  expected  that  the  student  could  take  any 
message  and  determine  which  of  the  four 
security  classifications  it  belongs  in. 

On  the  job,  he  will  be  required  to  apply  the 
guidelines  to  messages  he  has  not  seen  before. 
Presumably  the  messages  will  be  varied  enough 
so  that  doing  one  is  not  like  doing  them  all. 

The  given  guidelines  should  contain  characteristics 
of  messages  that  help  determine  the  type  of 
classification. 


From  the  examples  above,  it  should  be  clear  that  the  key  to 
classification  is  REMEMBER  THE  JOB.  Whenever  there  is  doubt  about 
the  classification  of  an  objective  or  test  item,  a  subject-matter 
expert  or  technical  manual  should  be  consulted,  so  that  information 
about  the  job  can  be  obtained. 
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Chapter  2 

OBJECTIVE  ADEQUACY 


Introduction 

Why  are  objectives  used  in  instructional  development?  The  reason 
is  communication.  Objectives  communicate  to  everyone  involved  in  an 
instructional  program— managers ,  designers,  writers,  instructors,  and 
the  students  themselves— what  the  program  is  meant  to  accomplish.  Each 
objective  should  specify  something  the  student  must  be  able  to  do  at 
the  end  of  the  course  that  he  couldn't  do  at  the  beginning.  If  the 
objective  does  not  communicate  this  clearly,  or  if  it  specifies  something 
inappropriate  for  the  intent  of  the  instructional  program,  then  it  is 
not  adequate. 

For  an  objective  to  communicate  clearly,  it  must  contain  three 
parts.  First,  it  must  specify  the  CONDITIONS  under  which  the  student 
is  to  perform.  Second,  it  must  specify  what  STANDARDS  the  performance 
must  meet.  Third,  it  must  specify  what  the  performance  is;  that  is, 
what  ACTION  the  student  is  to  perform.  These  three  parts  are  the 
minimum;  additional  information  might  have  to  be  provided  to  make 
the  objective  clear.  Remember,  the  objective  must  communicate  to 
test  developers  and  to  instructional  developers.  How  could  a  test 
developer  write  an  item  if  the  standards  were  not  known? 

A  good  check  on  whether  or  not  an  objective  is  clear  is  to  try 
to  classify  it  according  to  the  classification  scheme  in  chapter  1. 

If  an  objective  is  hard  to  classify  (if  it  is  hard  to  decide  which 
box  it  goes  in),  this  means  that  the  ACTION  is  unclear;  we  don't 
know  exactly  what  the  student  must  do. 

An  objective  may  be  clear,  but  be  inappropriate  for  the  intent 
of  the  instructional  program.  In  this  case,  the  objective  is  still 
inadequate.  To  be  appropriate,  an  objective  must  prepare  the  student 
for  what  he  will  be  required  to  do  or  know  following  the  instructional 
program.  This  following  duty  could  be  anything  from  job  performance, 
to  on-the-job  training,  to  another  formal  follow-on  school;  these  are 
all  "jobs"  after  a  training  program.  To  determine  appropriateness 
of  an  objective,  the  key  is  to  "REMEMBER  THE  JOB." 


On  the  next  two  pages,  the  OBJECTIVE  ADEQUACY  procedures  from  Volume  IV 
are  reproduced.  These  procedures  correspond  to  the  criteria  discussed 
above.  After  the  procedures,  additional  explanation  is  given.  Finally, 
the  procedures  are  applied  to  example  objectives. 


27 


1 


I 


OBJECTIVE  ADEQUACY 


STEP  1:  El  TER  the  COURSE  TITLE  and  OBJECTIVE  NUMBER  at  the  top  of  the  form. 

STEP  2:  Determine  whether  or  not  the  OBJECTIVE  is  CORRECTLY  STATED. 

2a:  Are  the  CONDITIONS  under  which  student  performance  is  expected  specified? 

mviPOUHEUT:  PHYSICAL  (weather,  tine  of  day,  lighting,  etc.) 

SOCIAL  (isolation,  individual,  team,  audience,  etc.) 
PSYCHOLOGICAL  (fatigue,  stress,  relaxed,  etc.) 

INFORMATION:  GIVEN  INFORMATION  (scenario,  formula,  values,  etc.) 

CUES  (signals  for  starting  or  stopping) 

SPECIAL  INSTRUCTIONS 


Instructional 

Quality 

Inventory 


CARO  1 


RESOURCES:  JOB  AIDS  (cards,  charts,  graphs,  checklists,  etc.) 

EQUIPMENT,  TOOLS 
TECHNICAL  MANUALS 


2b:  Are  the  STANDARDS  which  the  student  performance  must  meet  specified’ 

PERFORMANCE:  COMPLETENESS  (how  much  of  the  task  must  be  performed) 
ACCURACY  (how  well  must  each  task  be  performed) 

TIKE  LIMIT  (how  much  time  is  allowed) 

RATE  (how  fast  must  task  be  done) 

PRODUCT:  COMPLETENESS  (what  must  finished  product  contain) 

QUALITY  (what  objective  standard  must  product  meet) 

JUDGEMENT  (what  subjective  opinions  must  product  satisfy) 

2c:  Is  the  ACTION  the  student  must  perform  specified? 

Is  an  action  verb  used  to  specify  what  the  student  must  do? 

Is  only  one  action  stated  in  the  objective? 

STEP  3:  Determine  whether  or  not  the  OBJECTIVE  is  CLASSIFIABLE?  Does  the  OBJECTIVE  fit 
in  one  and  only  one  cell  of  the  table  below? 


FACT 

CATEGORY 

PROCEDURE 

RULE 

PRINCIPLE 

4  i 

f 

Ro'itnoER 

RECALL  OR  RE¬ 
COGNIZE  NAMES, 
PARTS,  DATES, 
PLACES,  VO¬ 
CABULARY  DEF¬ 
INITIONS,  ETC. 

REMEMBER  THE 
CHARACTERISTICS 
OF  EACH  CATE- 
GORYAND  THE  ^ 
GUIDELINES  FOR 
CLASSIFICATION 

REMEMBER  THE 
STEPS  OF  THE 
PROCEDURE. 

REMEMBER  THE 
FORMULA  OR 

THE  STEPS  OF 
THE  RULE, 

REMEMBER  THE 
CAUSE  AND 
EFFECT  RELA¬ 
TIONSHIPS  OR 
THE  STATEMENT 
OF  THE  PRIN¬ 
CIPLE. 

( 

,  A 

i 

\ 

-  > 

%  i 

> 

t 

USE  UNAIDED 

CLASSIFY  OR 
CATEGORIZE 
OBJECTS,  E- 
VEIITS,  IDEAS, 
ACCORDING  TO 
THEIR  CHARAC¬ 
TERISTICS, 

WITH  HO  MEMORY 
AID. 

APPLY  THE 

STEPS  OF  THE 
PROCEDURE  IN 

A  SINGLE  SIT¬ 
UATION  OR  ON 

A  SINGLE 

PIECE  OF 
EQUIPMENT, 

WITH  NO  MEM¬ 
ORY  AID. 

APPLY  THE 
FORMULA  OR 

RULE  TO  A 
VARIETY  OF 
PROBLEMS  OR 
SITUATIONS, 
WITH  NO  MEM¬ 
ORY  AID. 

USE  THE  PRIN¬ 
CIPLE  TO  IN¬ 
TERPRET  OR 
PREDICT  WHY 

OR  HOW  THINGS 
HAPPENED  OR 
WILL  HAPPEN, 
WITH  NO  MEM¬ 
ORY  AID, 

1 

i 

l 

7 

w 

1 

USE  AIDED 

GIVEN  CATEGORY 
CHARACTERIS¬ 
TICS'  AND  GUIDE¬ 
LINES,  CATE¬ 
GORIZE  OBJECTS, 
EVENTS,  IDEAS, 
ACCORDING  TO 
CHARACTERIS¬ 
TICS. 

GIVEN  STEPS  OF 
THE  PROCEDURE, 
APPLY  THE  PRO¬ 
CEDURE  IN  A 
SINGLE  SIT¬ 
UATION,  OR 

ON  A  SINGLE 
PIECE  OF 
EQUIPMENT , 

GIVEN  THE 
FORMULA  OR 
RULE  STEPS, 
APPLY  THE 
FORMULA  OR 
RULE  TO  A 
VARIETY  OF 
PROBLC  IS  OR 
SITUATl  NS. 

GIVEN  A  STATE¬ 
MENT  OF  THE 
PRINCIPLE, 
INTERPRET  OR 
PREDICT  WHY 

OR  HOW  THINGS 
HAPPENED  OR 
WILL  HAPPEN, 

1 
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STEP  4:  Determine  whether  or  not  the  OBJECTIVE  is  APPROPRIATE? 


4a:  Are  the  CONDITIONS  appropriate  for  the  work  to  be  performed  on  the  Job  or  for 
later  training? 

4b:  Are  the  STANDARDS  appropriate  for  the  work  to  be  performed  on  the  job  or  for 
later  training? 

4c:  Is  the  TASK  LEVEL  of  the  ACTION  appropriate  for  the  work  to  be  performed  on  the 
job  or  for  later  training? 

4d:  Is  the  CONTENT  TYPE  of  the  ACTION  appropriate  for  the  work  to  be  performed  on  the 
job  or  for  later  training? 

4e:  If  this  objective  is  REMEMBER,  is  there  a  later  USE  objective? 

4f:  If  this  objective  is  USE-UNAIDED,  is  there  a  previous  REMEMBER  objective? 

4g:  If  this  objective  is  USE-AIDED,  is  the  aid  adequate,  or  are  other  objectives 
on  the  aid  included? 

Note,  if  the  answer  to  4d,  4e,  or  4g  is  yes,  and  if  the  associated  objective  is 

be  taught  in  the  present  course,  evaluate  that  objective  next  and  keep  the  related  objectives 

togeUter  throughout ■  the  IQT  evaluation. 


STEP  1:  This  step  refers  to  the  Objective  Adequacy  Form  on  page  1  of 

volume  IV.  It  must  be  emphasized  that  this  form  is  not  required ; 
the  procedures  can  be  used  without  any  form.  This  form  is  only 
a  suggestion,  and  users  of  the  IQI  may  use  it,  or  design  their 
own,  or  use  no  foimi  at  all. 

STEP  2:  Steps  2a  and  2b  refer  to  the  CONDITIONS  avd  STANDARDS  parts  of 
an  objective.  Several  categories  of  conditions  and  standards 
are  given.  Obviously,  no  objective  will  require  all  of  these. 

Each  objective  should  be  reviewed  with  these  categories  in  mind, 
and  a  decision  should  be  made  about  whether  or  not  they  are 
applicable.  If  you  are  unsure  about  whether  or  not  a  particular 
condition  or  standard  should  be  included  in  an  objective,  a  good 
rule  is  "when  in  doubt,  stick  it  in. " 

/Is  experienced  instructional  developers  know,  many  objectives 
contain  "implicit  *'  coiiditions  or  standards,  like  "Given  paper  and 
pencil,  ..."  or  " with  100 %  accuracy."  It  is  up  to  each  organization 
using  the  IQI  to  decide  whether  or  not  to  include  these  obvious 
conditions  and  standards.  Whatever  the  policy,  though,  it  should 
be  explicit. 

Step  2c  refers  to  the  ACTION  part  of  an  objective.  Obviously, 
the  action  part  should  use  an  action  verb.  It  is  usually  best 
that  there  be  only  one  action  per  objective ;  if  there  is  more 
than  one  action,  the  objective  should  probably  be  split  up  into 
several  objectives. 

The  action  verb  deserves  special  attention.  It  should  always 
be  an  action  which  is  observable  and  measurable.  This  means  you 
should  be  able  to  tell  whether  the  student  did  it  or  not.  Action 
verbs  like  "appreciate"  or  "understand"  are  garbage  words ;  who 
knows  what  they  mean? 

STEP  3:  Refer  back  to  chapter  1  for  classifying  objectives.  If  you  can't 
classify  an  objective,  the  objective  needs  to  be  fixed. 

If  the  objective  fits  in  more  than  one  box,  it  probably 
needs  to  be  split  up  into  more  than  one  objective. 

STEP  4:  Steps  4a,  4b,  4c,  and  4d  all  mean  "REMEMBER  THE  JOB. "  The  intent 
of  any  course  is  to  prepare  the  student  to  do  something  after  he 
finishes  the  course.  This  "something"  is  the  JOB.  The  wrd  JOB 
incorporates  a  wide  range  of  activities,  including  on-the-job 
training ,  another  more  advanced  course,  or  actual  job  performance. 
Therefore,  the  CONDITIONS,  STANDARDS,  TASK  LEVEL,  and  CONTENT  TYPE 
should  be  carefully  evaluated,  to  make  sure  they  are  appropriate 
for  the  "job. " 
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Steps  4e,  4f,  and  4g  ax-e  a  partial  cheek  on  the  task  analysis 
that  led  to  the  objectives. 

Step  4e  means  that  there  is  no  point  teaching  someone  to  REMEMBER 
something  if  it  will  never  be  USED  later.  Note  that  ’’later”  may 
be  on  the  job  or  in  a  later  course. 

This  is  just  as  true  for  FACTS  as  it  is  for  the  other  content 
types,  but  FACTS  are  not  "used”  in  quite  the  same  way.  The 
information  taught  at  the  REMEMBER  level  for  CATEGORIES, 
PROCEDURES,  RULES,  and  PRINCIPLES  can  be  directly  USED,  but 
FACTS  cannot.  Instead,  FACTS  provide  information  to  support 
all  the  other  task/content  types.  The  classification  scheme 
has  been  redrawn  in  the  chard  below  to  show  this. 


FACTS 

RECALL  OR  RECOGNIZE  NAMES,  PARTS,  DATES,  PLACES,  VOCABULARY  DEFINITIONS,  ETC. 


REMEMBER 

_  -  -1 

REMEMBER  THE  CHAR¬ 
ACTERISTICS  OF 

EACH  CATEGORY  AND 
THE  GUIDELINES  FOR 
CLASSIFICATION. 

REMEMBER  THE  STEPS 
OF  THE  PROCEDURE. 

REMEMBER  THE  FORM¬ 
ULA  OR  STEPS  OF 

THE  RULE. 

REMEMBER  THE  CAUSE 

1  AND  EFFECT  RELA¬ 
TIONSHIPS  OR  THE 
STATEMENT  OF  THE 
PRINCIPLE. 

USE  UNAIDED 

CLASSIFY  OR  CATE¬ 
GORIZE  OBJECTS, 
EVENTS,  IDEAS, 
ACCORDING  TO  THEIR 
CHARACTERISTICS, 
WITH  NO  MEMORY  AID. 

APPLY  THE  STEPS 

OF  THE  PROCEDURE 

IN  A  S.NGLE  SIT¬ 
UATION  OR  ON  A 
SINGLE  PIECE  OF 
EQUIPMENT,  WITH  NO 
MEMORY  AID. 

APPLY  THE  FORMULA 

OR  RULE  TO  A 

VARIETY  OF  PROBLEMS 
OR  SITUATIONS, 

WITH  NO  MEMORY  AID. 

USE  THE  PRINCIPLE 

TO  INTERPRET  OR 
PREDICT  WHY  OR  HOW 
THINGS  HAPPENED  OR 
WILL  HAPPEN,  WITH 

NO  MEMORY  AID. 

USE  AIDED 

GIVEN  CATEGORY 
CHARACTERISTICS 

AND  GUIDELINES, 
CATEGORIZE  OBJECTS, 
EVENTS,  IDEAS  AC¬ 
CORDING  TO  CHAR¬ 
ACTERISTICS. 

GIVEN  STEPS  OF  THE 
PROCEDURE,  APP.’.Y 

THE  PROCEDURE  IN 

A  SINGLE  SITUATION,, 
OR  ON  A  SINGLE 

PIECE  OF  EQUIPMENT. 

GIVEN  THE  FORMULA 

OR  RULE  STEPS, 

APPLY  THE  FORMULA 

OR  RULE  TO  A  VAR¬ 
IETY  OF  PROBLEMS 

OR  SITUATIONS. 

GIVEN  A  STATEMENT 

OF  THE  PRINCIPLE, 
INTERPRET  OR  PRE¬ 
DICT  WHY  OR  HOW 
THINGS  HAPPENED  OR 
WILL  HAPPEN. 

|  CATEGORY 

PROCEDURE 

RULE 

PRINCIPLE 

Step  4e  for  FACTS  therefore  means  "Is  there  some  later  objective 
which  requires  that  the  student  know  that  fact  information?" 


Step  4f  is  the  reverse  of  4e.  If  an  objective  is  USE-UNAIDED,  this 
means  that  the  student  must  remember  what  to  do,  and  then  do  it. 
Therefore,  there  should  be  a  previous  objective  at  the  REMEMBER 
level.  Note  that  "previous"  may  be  in  an  earlier  course,  or  may 
even  be  an  entry  behavior  for  the  student. 


If  the  task  level  is  REMEMBER ,  special  care  must  be  taken  to 
make  sure  that  the  ACTION  is  appropriate.  The  reason  for  this 
is  that  there  are  really  two  kinds  of  remembering  —  recognition 
and  recall.  Recognition  involves  selecting  or  choosing  from 
given  alternatives,  or  matching  given  pieces  of  information, 
or  fudging  the  accuracy  of  a  given  statement.  In  recognition, 
all  the  information  is  given;  the  student  only  has  to  make 
a  decision  about  it.  Recall,  on  the  other  hard,  involves 
reproduci',ig  from  memory  some  piece  of  information.  Recognition 
and  recall  are  different  because  recall  involves  more  learning 
than  recognition. 

To  make  decisions  about  the  appropriateness  of  recognition  or 
recall,  you  must  REMEMBER  THE  JOB.  Most  job  situations  require 
recall.  For  example,  the  steps  of  a  USE-UNAIDED  procedure  must 
be  recalled  so  they  can  be  performed.  The  same  is  true  for 
using  categories ,  rules,  and  principles.  Many  facts  also  have 
to  be  recalled. 

There  are  two  situations  in  which  recognition  can  be  appropriate 
at  the  REMEMBER  level.  The  first  occurs  with  FACTS,  when  a 
selection  must  be  made  from  a  group  of  objects,  locations,  etc. 
For  example,  the  task  "go  to  the  tool  box  and  get  a  ball-peen 
hammer "  is  a  REMEMBER-FACT  recognition  task. 

The  second  situation  can  occur  for  any  content  type  at  the 
REMEMBER  level  when  the  job  only  requires  the  student  to  be 
generally  familiar  with  the  remember  level  information. 

This  situation  only  happens  when  the  student  is  being  prepared 
for  later  on-the-job  or  formal  training,  and  even  then,  only 
when  the  student  will  be  closely  supervised.  This  is  because 
the  supervisor  can  take  care  of  memory  failure.  For  example , 
a  student  performing  the  steps  of  a  maintenance  procedure  on 
a  piece  of  equipment  may  not  need  to  have  memorized  the  steps 
if  his  supervisor  was  available  to  correct  any  errors  or  to 
tell  him  what  to  do  next.  Even  in  this  situation,  it  would 
have  been  more  efficient  and  more  consistent  with  the  job  if 
the  student  had  been  required  to  recall  the  steps  of  procedure 
during  training.  If  training  time  is  limited ,  the  recall 
performance  criterion  may  be  lowered  when  the  job  only  requires 
"general  familiarity. "  That  is,  students  may  be  permitted  to 
perform  at  less  than  100%. 

The  distinction  between  recognition  and  recall  is  important, 
because  the  type  of  testing  to  be  done  later,  and  the  type 
of  instruction  to  be  provided,  depends  on  whether  the  student 
is  being  trained  to  the  recognition  or  the  recall  level. 
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Step  4g  is  a  cheek  on  the  quality  of  the  Job  Performance  Aid. 

There  are  a  lot  of  terrible  Job  Performance  Aids  in  the  world  of 
work.  AlsOj  Job  Performance  Aids  may  be  hard  to  find}  or  hal'd 
to  use3  or  may  use  different  technical  vocabulary.  In  these  cases 3 
you  may  want  to  include  objectives  on  how  to  use  the  aids 3  or  how 
to  find  them 3  or  additional  fact  objectives  on  the  vocabulary. 

In  the  worst  ease 3  the  Aid  may  be  so  bad  that  the  objective  must 
be  rewritten  as  USE-UNAIDED.  (Then  you  would  also  need  a  previous 
REMhBER  objective. ) 


Objectives  Must  Communicate 


At  the  beginning  of  this  chapter 3  we  said  that  objectives  must 
communicate  to  test  developers 3  instructional  developer s3  instructors 3 
etc.  This  corrmmication  purpose  must  always  be  kept  in  mind  when 
reviewing  objectives3  especially  when  subject-matter  expertise  is  not 
readily  available.  It  is  often  desirable  to  include  very  specific3 
detailed  descriptions  of  conditions3  standards3  and  actions3  so  that 
later  misunderstanding  or  errors  do  not  occur. 

For  example 3  a  complete  objective  may  have  to  include  not  just 
a  concept3  but  a  complete  list  of  critical  characteristics;  all  steps 
of  a  procedure  or  rule  might  have  to  be  shown;  a  detailed  description 
of  a  principle  might  be  necessary.  This  could  be  accomplished  by 
including  references  to  documents  that  contain  this  information. 

The  reason  this  might  be  necessary  is  that  misunderstandings  may 
occur  if  the  instructional  development  team  does  not  have  sufficient 
subject-matter  expertise.  For  example 3  a  test  developer  may  have  to 
see  all  steps  of  a  procedure  to  write  a  good  test  item  on  it3  or  to 
specify  good  scoring  criteria. 


EXAMPLES 


In  this  section,  the  sample  objectives  from  chapter  1  are 
repeated  in  the  same  order.  This  time,  they  are  rated  for  objective 
adequacy.  For  each  objective,  we  will  first  determine  whether  or 
not  it  is  correctly  stated,  then  whether  or  not  it  is  classifiable 
(we  will  refer  to  chapter  1),  and  finally  whether  or  not  it  is 
appropriate.  The  examples  are  formatted  as  shown  below,  and  our 
assumptions  about  conditions,  standards,  and  appropriateness  are 
indicated. 

OBJECTIVE: 


Correctly  Conditions?  _ 

Stated? 

/4s  we  mentioned  earlier  in  this  chapter,  many  objectives 
contain  " implicit "  or  implied  conditions  and  standards.  Unless 
otherwise  stated  in  the  objective,  we  will  assume  that  the  student 
is  to  perform  under  normal  classroom  conditions,  is  given  paper 
and  pencil,  and  is  given  appropriate  instructions. 

Standards? 


Unless  otherwise  stated,  we  will  assume  that  the  student  will 
write  all  responses,  and  that  the  responses  must  be  "100%  correct. " 
Therefore,  when  an  objective  says  "state"  or  "recall,"  we  will 
assume  this  means  "write  from  memory  with  100 %  accuracy. " 

Action?  _ 


Classifiable?  Task  Level? 


Content  Type?  _ 

For  these  decisions ,  we  will  often  refer  to  chapter  1. 

Appropriate?  Conditions?  _ 

Standards? 


Action? 


In  these  examples,  we  have  collapsed  steps  4c  (task  level  of 
action  appropriate ?)  and  4d  (content  type  of  action  appropriate?) 
into  one  step,  "is  the  ACTION  appropriate?" 

Previous  Later  Aid 

Remember?  Use-Unaided?  Adequate? 

These  objectives  are  taken  "out  of  context. "  Therefore,  some  of 
the  appropriateness  decisions  will  be  difficult  to  make.  In 
general,  we  will  assume  that  these  objectives  ore  from  apprentice- 
level  courses. 
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1.  OBJECTIVE:  "The  student  will  state  the  rule  for  finding  total 
inductance  in  a  series  circuit." 

Correctly  Conditions?  IMPLIED _ 

Stated? 

Standards?  IMPLIED _ 

In  this  objective,  our  assumptions  about  implied  conditions  and 
standards  apply.  He  are  assuming  that  the  student  will  state  in 
writing  from  memory ,  with  100 %  accuracy. 

Action?  Q.K.  -  ONE  ACTION 

This  objective  uses  an  action  verb,  "state. "  There  is  only  one  action. 

Classifiable?  Task  Level?  REMEMBER _ 

Content  Type?  RULE _ 

See  Chapter  1.  This  objective  is  clearly  Remeinber  Rule. 

Appropriate?  Conditions?  Q.K. _ 

For  nearly  all  Remember-level  objectives,  the  performance  expected 
is  some  written  or  oral  response  under  fairly  normal  classroom 
conditions.  Occasionally,  other  conditions  might  apply,  such  as 
fatigue,  noisy  environment,  etc.  These  conditions  would  have  to  be 
stated.  The  key  is  REMEMBER  THE  JOB. 

Standards?  Q.K. _ 

For  nearly  all  Remember-level  objectives,  the  implied  standard  of 
100 %  accuracy  is  appropriate.  There  is  no  point  in  having  someone 
remember  something  unless  it  really  has  to  be  remembered.  The  only 
exception  is  in  a  familiarization  situation ,  where  for  cost  reasons 
the  standard  may  be  slightly  relaxed. 

Action?  Q.K. _ 

Later 

Use-Unaided?  __? _ _ 

This  objective  is  out  of  context.  For  a  whole  course,  we  would 
expect  to  see  a  later  Use-unaided  objective  which  requires  the 
student  to  calculate  total  inductance  in  series  circuits. 

REVISED  This  objective  is  O.K.  If  implicit  conditions  and 

OBJECTIVE:  standards  were  not  allowed,  the  objective  would  be: 

"The  student  will  write  from  memory  without  error  the 
rule  for  . . . . " 
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ranks  of  Naval  officers,  the  student  will  identify 
the  ranks  they  represent." 

Correctly  Conditions?  INCOMPLETE 
Stated? 

Some  conditions  are  necessary  and  cannot  he  implicit ,  because 
the  student  must  he  given  something  to  identify.  The  problem  here 
is  that  we  don't  know  what  the  pictures  contain.  Is  there  a  single 
collar  device  in  each  picture?  Are  the  collar  devices  pictured 
by  themselves ,  or  are  they  attached  to  a  uniform?  Are  the  devices 
from  the  U.S.  Navy,  or  other  navies?  Are  they  hard  to  see  on  the 
pictures?  Are  there  any  other  cues  as  to  rank? 

Standards?  IMPLIED 


Action?  O.K.  -  ONE  ACTION 


The  implied  standard  is  100%  accuracy.  The  action  is  "identify." 
Classifiable?  Task  Level?  REMEMBER 


Content  Type?  FACT _ 

See  Chapter  1.  The  student  must  learn  each  collar  device  and 
the  rank  it  stands  for. 

Appropriate?  Conditions?  ? _ 


What  is  the  JOB?  Ultimately,  the  student  will  have  to  identify 
Navy  officers'  ranks  by  looking  at  their  uniforms.  Rank  can  be 
identified  by  collar  devices,  sleeve  stripes ,  and  shoulder  boards, 
not  all  of  which  appear  on  any  one  uniform.  All  of  these  should 
be  included  in  the  objective.  Next,  for  this  training  situation, 
are  pictures  appropriate?  They  can  be,  if  they  are  similar  to 
situations  that  will  be  encountered  on  the  job.  Therefore,  they 
should  contain  collar'  devices,  sleeve  stripes,  or  shoulder  boards, 
on  actual  officers'  uniforms,  and  should  be  taken  from  the  point 
of  view  of  the  student.  They  should  portray  typical  situations 
involving  interactions  with  officers. 

Standards?  O.K. 


100%  accuracy  'is  ccpvropriaic. 

Action?  UNCLEAR 


"Identify"  is  fuzzy.  It  would  be  better  to  put  either  "write  from 
memory"  or  "state  orally  from  memory"  the  rank  associated  with 
each  .... 

Later 

Use-Unaided  ? 


REVISED  "Given  a  separate  picture  for  each  U.S.  Navy  officer 

OBJECTIVE:  uniform,  showing  collar  devices,  shoulder  boards,  or 

sleeve  stripes,  in  a  typical  job  environment,  the  student 
will  state  orally  from  memory  the  rank  associated  with 
each  collar  device,  shoulder  board,  or  sleeve  stripe." 
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3.  OBJECTIVE:  "Given  any  resistor  with  four  color  bands,  the  student 

will  state  the  ohmic  value  indicated  by  the  color  bands." 

Correctly  Conditions?  SPECIFIED 

Stated? 

Standards?  IMPLIED _ 

Action?  O.K.  -  ONE  ACTION 

The  condition  is  " given  any  resistor , "  the  standard  is  implied- 
100 %  accuracy ,  and  the  action  is  "state  the  ohntic  value. " 

Classifiable?  Task  Level?  USE-UNAIDED 

Content  Type?  PROCEDURE _ _ 

See  Chapter  1. 

Appropriate?  Conditions?  O.K. _ 

The  job  involves  determining  the  ohmic  value  of  resistors ;  therefore , 
the  conditions  are  appropriate.  If  resistors  were  not  available , 
pictures  might  be  substituted,  if  they  were  representative  of  the  job. 

Standards?  O.K. _ 

The  standards  could  be  less  than  100%  if  putting  the  wrong  value 
resistor  in  a  circuit  is  not  dangerous  or  costly,  however,  this 
is  not  the  case. 

Action?  UNCLEAR _ 

The  verb  "state"  is  misleading  for  the  reasons  discussed  in  Chapter  1. 
Previous 

Remember  _ ? _ 

This  objective  is  out  of  context.  Again ,  we  would  expect  to  see 
an  objective  on  recall  of  the  procedure  for  calculating  ohmi-c 
value  either  earlier  in  this  course,  or  in  an  earlier  course. 

REVISED  "Given  any  resistor  with  four  color  bands,  the  student 

OBJECTIVE:  will  follow  the  procedure  for  determining  ohmic  value 

as  indicated  by  the  color  bands." 


4.  OBJECTIVE:  "The  student  will  write  from  memory  the  steps  of  the 
procedure  for  message  reception  and  duplicate  checking 
as  listed  in  the  current  edition  of  NTP-4." 


Correctly 

Stated? 

Conditions? 

IMPLIED 

Standards? 

SPECIFIED 

Action? 

O.K.  -  ONE  ACTION 

The  standard  is  "as  listed 

in  the  current  edition  of  UTF-4. 

CLASSIFIABLE? 

Task  Level? 

REMEMBER 

Content  Type? 

PROCEDURE 

APPROPRIATE? 

Conditions? 

O.K. 

Standards? 

O.K. 

Action? 

O.K. 

Later 

Use-Unaided? 

? 

There  should  be  a  later  objective  requiring  the  student  to  receive 
messages  and  check  for  duplicates. 


REVISED 

OBJECTIVE: 


G.Y. 


5.  OBJECTIVE: 


Correctly 

Stated? 


"The  student  will  describe  the  principles  of  operation 
of  a  rotary  gear  pump." 


Condi tions? 
Standards? 


IMPLIED 


INCOMPLETE 


With  Px-ijzciple  objectives,  particular  care  must  be  given  to  specifying 
the  standards.  In  t'nis  case,  the  implied  standard  is  "accurately," 
but  we'don’t  knew  what  " accurately "  means.  The  solution  to  this 
problem  is  to  specify  completely  vhat  information  the  student  's 
description  must  contain.  This  can  be  done  eitnev  in  the  objective , 
or  by  reference  to  some  other  document. 


Action? 

Classifiable?  Task  Level? 


O.K.  -  ONE  ACTION 


Content  Type?  PRINCIPLE _ 

In  chapter  1,  we  discussed  the  reasons  why  this  objective  is 
difficult  to  classify.  The  main  problem  with  the  "describe" 
action  verb  is  that  it  is  noc  precise  enough  to  determine  the 
task  level.  At  this  point  in  the  objective  adequacy  procedure, 
the  action  should  be  revised. 


Appropriate?  Conditions? 

Standards? 


O.K. 


The  standards  should  be  revised  as  discussed  above,  and  care  should 
be  taken  to  make  sure  that  they  are  appropriate  for  the  "job.  " 

Action?  ? 


The  revised  action  should  also  be  appropriate  for  the  "job. " 
Let’s  assume  that  recall  of  the  principle  is  required. 

Later 

Use-Unaided?  ? 


If  this  objective  is  Remember,  then  there  should  be  a.  later  objective 
requiring  the  student  to  use  the  principle  to  predict  something 
about  the  pump's  operation. 

REVISED  "The  student  will  write  from  memory  the  principles 

OBJECTIVE:  of  operation  of  a  rotary  gear  pump,  as  described  in 

Tech.  Manual  XXX." 
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6.  OBJECTIVE:  "Given  a  globe  valve,  rags,  prussian  blue,  gasket 

material,  packing,  tools,  and  an  MRC  job  program  card, 
the  student  will  disassemble  and  reassemble  the 
globe  valve." 

Correctly  Conditions?  SPECIFIED _ 

Stated? 

Standards?  INCOMPLETE 

The  reason  the  standards  are  incomplete  is  that  there  may  be 
a  time  limit  for  performing  the  task.  If  there  is-  then  it 
should  be  specified  in  the  objective.  If  time  is  not 
important  then  the  implicit  standards  are  O.K. 

Action?  O.K. _ 

Although  there  are  two  action  verbs ,  assemble  and  disassemble, 
the  action  is  all  right  because  the  verbs  refer  to  different 


steps  in  a 

procedural  task. 

Classifiable? 

Task  Level? 

USE-AIDED 

Content  Type? 

PROCEDURE 

See  Chapter  l. 

Appropriate? 

Conditions? 

O.K. 

It  is  assumed  that  all  the  materials  listed  In  the  objective  will 
be  available  on  the  job. 

Standards?  __? _ 

Standards  are  O.K.  if  no  time  limit  is  required. 

Action?  O.K. _ 

Aid 

Adequate?  _? _ 

If  the  MRC  card  has  unfamiliar  terms  or  instructions  on  it,  then 
there  should  be  additional  objectives  to  clarify  the  aid. 

REVISED  This  objective  is  O.K.  unless  a  time  standard  needs 

OBJECTIVE:  to  be  specified.  A  subject-matter  expert  should  be 

consulted  to  make  this  decision. 


7.  OBJECTIVE: 


"The  student  will  classify  any  call  sign  according 
to  one  of  the  following  types  --  international  U.S. 

Navy  ship,  international  U.S.  Navy  shore,  indefinite 
task  organization,  voice,  or  not  a  valid  Navy  cal'!  sign." 

Correctly  Conditions?  UNCLEAR _ 

Stated? 

It  is  not  clear  whether  or  not  the  student  will  be  given 
the  names  of  the  types  of  call  signs  during  the  classification. 


Standards? 

IMPLIED 

Action? 

O.K.  -  ONE  ACTION 

Classifiable? 

Task  Level? 

USE-UNAIDED 

Content  Type? 

CATEGORY 

Appropriate? 

Conditions? 

? 

On  the  gob ,  does  the  student  have  to  classify  call  signs 
without  having  the  names  of  the  types  available?  Also, 
the  conditions  should  specify  any  special  situations  that 
o:re  typical  of  the  job . 

Standards? 

O.K. 

Action? 

O.K. 

Previ ous 
Remember? 

? 

There  should  be  a  previous  Remember  objective  that  requires  the 
student  to  memorize  the  definitions  or  critical  characteristics 
of  each  type  of  call  sign. 


REVISED  "The  student  will  classify  any  call  sign  according 

OBJECTIVE:  to  one  of  the  following  types  --  international  U.S. 

Navy  ship,  international  U.S.  Navy  shore,  indefinite, 
task  organization,  voice,  or  not  a  valid  Navy  call  sign. 
Type  names  will  not  be  provided  during  classification.' 
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8.  OBJECTIVE:  "Given  the  formula  for  Ohm's  Law  and  two  values, 
the  student  will  solve  for  the  third  value." 

Correctly  Conditions?  INCOMPLETE _ _ 

Stated? 

The  conditions  should  specify  the  type  and  difficulty  of 
the  problems  to  be  solved.  In  addition ,  because  Ohm's  Law 
can  be  written  in  several  different  ways 3  the  conditions 
should  specify  which  form  or  forms -  of  the  formula  will  be  given. 

Standards?  INCOMPLETE _ 

The  number  of  decimal  places  should  be  specified. 

Action?  O.K.  -  ONE  ACTION 

Classifiable?  Task  Level?  USE-AIDED _ 

Content  Type?  RULE _ 

See  Chapter  1. 

Appropriate?  Conditions?  __? _ 

The  type  and  difficulty  of  the  problems  should  be  representative 
of  the  Job. 


Standards?  _ ? _ 

The  number  of  decimal  places  should  be  typical  of  what  is 
required  on  the  gob. 

Action?  O.K. _ 

Aid 

Adequate?  ? _ 


The  symbols  used  m  the  gormula  should  have  been  defmed  previous 
This  would  have  been  most  effectively  covered  in  a  fact-level 
objective. 


REVISED  "Given  the  formula  for  Ohm's  Law,  E=IR,  and  any  two 

OBJECTIVE:  values  typical  of  the  values  encountered  on  the  job, 

the  student  will  solve  for  the  third  value  correct  to 
two  decimal  places." 
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9.  OBJECTIVE:  "The  student  will  recall  the  duties  of  a  gun  captain." 


Correctly 

Conditions? 

IMPLIED 

Stated? 

Standards? 

IMPLIED 

Action? 

O.K.  -  ONE  ACTION 

Classifiable? 

Task  Level? 

REMEMBER 

Content  Type? 

As  discussed  in  chapter  lj  this  could  he  either  Fact  or 
Procedure.  The  objective  must  be  rewritten  to  clarify 
exactly  what  the  duties  involve. 

Appropriate? 

Conditions? 

O.K. 

Standards? 

O.K. 

Action? 

See  Above 

Later 

Use-Unaided? 

? 

There  should  be  a  later  Use-unaided  objective  which  requires 
the  student  to  perform  the  "duties. " 

REVISED  "The  student  will  recall  in  the  proper  order 

OBJECTIVE:  the  procedures  performed  by  a  gun  captain  while 

firing  during  general  quarters." 
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10.  OBJECTIVE:  "In  a  classroorfi  exercise,  the  student  will  file 
thirty  drill  messages  representing  Top  Secret, 
Secret,  special  category,  readdressed  general 
messages,  and  other  classified  and  unclassified 
messages  in  the  correct  files." 

Correctly  Conditions?  SPECIFIED 

Stated? 

Standards?  INCOMPLETE _ 

The  standard  given  is  "thirty  messages. . .in  the  correct  files." 
There  should,  however,  be  a  time  standard  for  those  thirty 
messages.  Furthermore,  rate  standards  like  this  really  should 
be.  written  as  "time  allowed  per  message." 

Action?  O.K.  -  ACTION 

Classifiable?  Task  Level?  USE-UNAIDED _ 

Content  Type?  ___? _ 

4s  discussed  in  chapter  1,  the  could  be  either  Category  or 
Procedure.  The  objective  should  be  rewritten  to  clarify  this. 

Appropriate?  Conditions?  __? _ 

If  the  classroom  exercise  situation  is  similar  to  the  job, 
then  the  conditions  are  appropriate.  If  this  is  not  true,  the 
objective  should  be  revised. 

Standards?  __? _ 

The  revised  standards  should  correspond  to  the  job  situation. 

Action?  _ ]_  _ 

The  revised  action  should  reflect  what  is  done  on  the  job. 

* 

Previous 

Remember?  __? _ _ 

There  should  be  a  previous  objective  for  this  task  at  the 
Eemember  level. 

REVISED  "In  a  classroom  exercise  simulating  shipboard 

OBJECTIVE:  working  conditions,  the  student  will  file  drill 

messages  identified  as  Top  Secret,  Secret,  special 
category,  readdressed  general  messages,  and  other 
classified  and  unclassified  messages,  in  the 
correct  files,  at  the  rate  of  one  message  every 
60  seconds." 
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11.  OBJECTIVE: 

"Given  schematic  symbols  for  common  electronic 
components,  the  student  will  write  from  memory 
the  name  of  the  component  represented  by  each 
symbol 

Correctly 

Conditions? 

SPECIFIED 

Stated? 

Standards? 

IMPLIED 

Action? 

O.K.  -  ONE  ACTION 

Classifiable? 

Task  Level? 

REMEMBER 

Content  Type? 

FACT 

Appropriate? 

Conditions? 

C.K. 

We  will  assume  that  the  "common  electronic  components"  are 
ones  thaJ ■  are  typical  of  the  equipment  the  student  will  be 
trained  to  maintain  or  repair.  If  there  were  special  com¬ 
ponents  or  symbols  that  the  student  had  to  knoWj  then  these 
should  be  included. 

Standards? 

O.K. 

Action? 

O.K. 

Later 

Use-Unaided? 

? 

There  should  be  a  later  Use  objective  for  which  the  student 
must  be  able  to  recall  component  names  given  component  symbols. 

REVISED  O.K. 

OBJECTIVE: 
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12.  OBJECTIVE:  "Given  the  explanation  of  the  principle  of  supply 
of  a  military  force  from  the  text,  the  student 
will  discuss  how  this  principle  applies  in 
Hitler's  attack  on  Russia,  the  Battle  of  Midway, 
the  Battle  of  the  Bulge,  and  Sherman's  march 
through  Georgia." 

Correctly  Conditions?  SPECIFIED _ 

Stated? 

Standards?  IMCOMPLETE 


The  implied  standard  is  "correctly"  but  with  principle  objectives 
it  is  often  difficult  to  tell  what  correctly  means.  Therefore , 
either  the  correct  explanation  should  be  included  in  the  objective , 
or  a  document  containing  the  correct  explanation  should  be  referenced. 


Action? 

Classifiable?  Task  Level? 

Content  Type? 
See  Chapter  1. 
Appropriate?  Conditions? 

Standards? 

Action? 


O.K.  -  ONE  ACTION 

USE-AIDED _ 

PRINCIPLE 


? 


See  above. 


? 


The  appropriateness  questions  depend  on  how  the  student  is 
going  to  use  the  principle  on  the  job.  If  the  job  is  to 
plan  military  operation  given  reference  materials, ,  then  the 
conditions  are  probably  appropriate.  If,  however,  the  job 
involves  making  quick  decisions  on  the  battle  field,  then 
reference  materials  would  not  be  appropriate ,  and  the  task 
level  should  then  be  Use-Unaided. 


The  action  of  discussing  how  the  principle  of  supply  applied 
in  previous  battles  may  or  may  not  be  appropriate,  depending  on 
whether  or  not  the  principle  still  holds  in  modem  warfare. 


Aid 

Adequate?  __? _ 

Is  the  explanation  given  in  the  text  sufficient  to  allow 
the  student  to  give  a  correct  discussion,  or  should  other 
objectives  on  the  aid  be  included? 

REVISED  The  standard  should  be  revised  as  indicated 

OBJECTIVE:  above.  Other  revision  would  depend  on  answers 

to  the  appropriateness  questions  we  have  raised. 
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13.  OBJECTIVE: 


"The  student  will  recall  in  writing  the  lofargram 
characteristics  for  the  following  types  of  contacts 
merchant  ship,  aircraft  carrier,  destroyer,  whale 
diesel  submarine,  nuclear  submarine." 

Correctly  Conditions?  IMPLIED _ 

Stated? 

Standards?  IMPLIED _ 

Action?  OX  -  ONE  ACTION 

Classifiable?  Task  Level?  REMEMBER _ 

Content  Type?  CATEGORY _ 

Appropriate?  Conditions?  O.K. _ 

Standards?  O.K. _ 

Action?  O.K. _ 

Later 

Use-Unaided?  ?  _ _ 

There  should  be  a  later  Use-unaided  objective  which  requires 
to  categorize  lofargrams  according  to  the  characteristics 
to  determine  type  of  contact. 

REVISED  O.K. 

OBJECTIVE: 
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14.  OBJECTIVE:  "The  student  will  solve  for  inductive  reactance  in 
a  circuit,  give  frequency  and  inductance." 

Correctly  Conditions?  UNCLEAR _ 

Stated? 

Is  the  student  given  the  circuit,  or  just  the  values 
for  frequency  and  inductance?  If  he  is  given  the  circuit , 
is  it  the  actual  equipment,  or  a  schematic? 

Standards?  IMPLIED _ 

Action?  O.K.  -  ONE  ACTION 

Classifiable?  Task  Level?  USE-UNAIDED 

Content  Type?  RULE _ 

Appropriate?  Conditions?  __? _ 

The  revised  conditions  should  he  appropriate  for  the  job. 

Standards?  O.K. _ 

Action?  O.K. _ 

Previous 

Remember?  _ ? _ 

There  should  be  a  previous  objective  requiring  the  student 
to  recall  the  formula  for  solving  for  inductive  reactance. 

REVISED  "Given  a  circuit  schematic  with  values  of 

OBJECTIVE:  frequency  and  inductance  specified,  the 

student  will  solve  for  inductive  reactance." 


f 
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15.  OBJECTIVE:  "The  student  will  perform  the  steps  required  to 

accept,  verify,  and  log  messages  to  be  transmitted 
via  teletype  tape." 

Correctly  Conditions?  INCOMPLETE _ 

Stated? 

The  conditions  should  specify  the  equipment  the  student  mill  use  and 
the  environmental  conditions. 

Standards?  INCOMPLETE _ 

A  time  or  rate  standard  may  be  required.  A  subject-matter  expert 
should  be  consulted. 

Action?  ?  -  MULTIPLE  ACTION 

If  accept ,  verify ,  and  log  are  all  steps  of  a  procedure  for 
transmitting  messages.,  then  the  action  is  0.  K.  If,  however , 
one  or  more  of  the  action  verbs  involves  a  different  content 
type,  then  additional  objectives  should  be  written. 

Classifiable?  Task  Level?  USE- UNA I PEP 

Content  Type?  PROCEPURE _ 

See  Chapter  1. 

Appropriate?  Conditions?  _? _ 

Standards?  __? _ 

Action?  __? _ 

Any  revisions  made  to  the  conditions,  standards,  and  action  should 
be  representative  of  the  job. 

Previous 

Remember?  __? _ 

There  should  be  a  previous  remember  objective  that  requires 
the  student  to  memorize  the  steps  of  the  procedure. 

REVISEP  "In  a  classroom  laboratory  simulating  a  ship 

OBJECTIVE:  radio  room,  the  student  will  perform  the  steps 

required  to  process  (accept,  verify,  log)  messages 
to  be  transmitted  via  teletype  tape,  at  the  rate 
of  5  minutes  per  message.” 
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16.  OBJECTIVE:  "The  student  will  state  the  principles  of  operation 
of  a  jet  pump  as  described  in  Propulsion  Engineering, 
Mod  6,  Lesson  2." 


Correctly  Conditions?  IMPLIED 
Stated? 

Standards?  ? 


There  is  a  subtle  problem  with  th^  standard.  This  objective 
is  for  the  propulsion  engineering  course ,  and  will  be  used  to 
develop  the  lesson  material  —  mod  6,  lesson  2.  Therefore, 
the  standard  does  riot  really  exist  yet.  Pre-existing  pub¬ 
lications  can  be  used  in  standards,  but  material  that  Ivxs 
not  yet  been  developed  caivnot. 


Action? 

Classifiable?  Task  Level? 

Content  Type? 

Appropriate?  Conditions? 

Standards? 

Action? 

I  3  fpr 

Use-Unaided? 


O.K.  -  ONE  ACTION 

REMEMBER _ 

PRINCIPLE _ 

O.K. _ 

See  Above _ 

O.K. _ 


? 


There  should  be  a  later  objective  requiring  the  student 

to  use  his  knowledge  of  how  the  pump  works  to  give  explanations 

or  predictions  about  pump  operation. 

REVISED  "The  student  will  state  the  principles  of  operation 

OBJECTIVE:  of  a  jet  pump  as  described  in  Technical  Manual  A-xxx." 
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17.  OtUECTIVF:  "Tha  student  will  ir,i>  the  principle  of  electro¬ 
magnetic  indi  <.lio>!.  to  de*.  .tribe  the  operation  of 
a'  AC  generator.'1 

Correctly  Conditions?  IMPLIED _ 

Stated? 

Standards?  INCOMPLETE 


The  standard  must  specify  what  the  student's  description  must 
contain  in  order  to  be  correct. 


Classifiable? 

Appropriate? 


Action? 

Task  Level? 
Content  Type? 
Conditions? 
Standards? 
Action? 


O.K.  -  ONE  ACTION 
USE-UNAIDED 

PRINCIPLE _ 

O.K. _ 

See  Above. _ 

? 


On  the  job,  will  the  student  ever  have  to  use  his  knowledge 
of  electromagnetic  induction  to  describe  the  operation  of 
equipment?  If  so,  then  this  objective  is  appropriate. 

On  the  other  hand ,  if  the  intent  was  just  to  have  the  student 
learn  the  principles  of  operation  of  the  AC  generator,  then 
the  objective  should  be  to  n Remember "  the  principles  of  operation. 
As  it  stands ,  the  objective  might  be  appropriate  if  the  student 
will  be  required  to  troubleshoot  new  AC  generators . 

Previous 

Remember?  ? 


There  should  be  an  earlier  objective  requiring  the  student 
to  remember  the  principle  of  electromagnetic  induction. 

REVISED  "The  student  will  use  the  principle  of  electro- 

OBJECTIVE:  magnetic  induction  to  describe  the  operation  of 

an  AC  generator.  The  description  must  contain 
the  following  points..." 


i 
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18.  OBJECTIVE:  "Given  the  guidelines  for  determining  message 

security  classification,  the  student  will  determine 
the  security  classification  (Top  Secret,  Secret, 
Confidential,  or  Unclassified)  for  outgoing 
messages." 

Correctly  Conditions?  _ SPECIFIED _ 

Stated? 

Standards?  IMPLIED _ 

Action?  0. K.  -  ONE  ACTION 

Classifiable?  Task  Level?  USE-AIDED _ 

Content  Type?  CATEGORY _ 


See  Chapter  1. 

Appropriate?  Conditions?  _ O.K. 

Standards?  0. K. 

Action?  O.K. 

Aid 

Adequate?  ? 


The  guidelines  should  he  checked  to  make  sure  they  are 
complete 3  and  are  detailed  enough  so  that  the  student 
can  use  them  to  perform  the  task. 

REVISED  This  objective  is  probably  O.K.  if  the  aid 
OBJECTIVE:  is  adequate. 
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Chapter  3 

TEST  CONSISTENCY  and  TEST  ADEQUACY 


Introduction 


At  this  point,  the  objectives  for  the  instructional  program 
have  been  checked  for  adequacy  and  revised  if  necessary.  The 
next  step  is  to  check  the  test  items,  to  make  sure  they  are 
consistent  with  objectives,  and  are  adequate. 

In  any  instructional  program,  each  test  item  should  be 
tied  or  "referenced"  to  a  specific  objective.  Since  each  ob¬ 
jective  is  a  statement  of  a  "criterion"  for  student  performance, 
tests  for  the  objectives  are  called  "criterion-referenced"  tests. 

A  test  item  is  "referenced"  to  an  objective  (criterion),  when 
it  is  consistent  with  the  objective.  Consistency  means  that  the 
conditions  and  standards  in  the  objective  must  be  maintained  in  the 
testing  situation.  Also,  the  task/content  of  the  test  item  must 
match  the  task/content  of  the  objective.  Finally,  the  format  of 
the  test  item  must  be  appropriate  for  the  task/content  classification 
of  the  objective. 

A  test  item  can  be  consistent  with  an  objective  but  still  be 
a  bad  item.  An  adequate  item  is  one  which  is  clear,  unambiguous, 
well -constructed,  and  free  of  "hints."  Also,  for  an  objective, 
there  must  be  enough  items  to  test  the  objective  adequately,  and 
the  student  must  be  given  the  opportunity  on  the  test  to  make 
errors  that  are  commonly  made  on  the  job. 


On  the  following  pages,  the  TEST  CONSISTENCY  and  TEST  ADEQUACY 
procedures  from  Volume  IV  are  reproduced.  Additional  explanation 
about  the  procedures  is  given.  Later,  the  procedures  are  applied 
to  example  test  items. 
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Instructional 

Quality 

Inventory 


TEST  CONSISTENCY 

CARD  2 

STEP  1:  ENTER  the  COURSE  TITLE  and  OBJECTIVE  NUMBER  at  the  top  of  the  form. 

STEP  2:  ENTER  the  TEST  ITEM  NUMBERS  for  the  items  associated  with  this  objective  on  the  form. 

STFP  S-  OE.TERMJNE  whether  or  not  the  CONDITIONS  in  each  item,  or  the  CONDITIONS  under  which  the 
Items  are  administered,  match  the  conditions  in  the  objective. 

STEt’  4:  DETERMINE  whether  or  not  the  STANDARDS  in  each  item,  or  the  STANDARDS  for  scoring  each 
Item,  match  the  standards  in  the  objective. 

STEP  5:  DETERMINE  whether  or  not  the  ACTIONS  in  each  item  match  the  action  of  the  objective. 

STEP  5a:  Deteimine  the  TASK  LEVEL  and  CONTENT  TYPE  of  each  test  item. 

STEP  5b:  Determine  whether  these  match  the  TASK  LEVEL  and  CONTENT  TYPE  of  the  objective. 

STEP  6:  DETERMINE  whether  or  not  each  item  is  typical  of  the  job  to  be  performed  after  training, 
or  is  a  necessary  qualification  for  later  training. 

STEP  7:  DETERMINE  whether  or  not  the  FORMAT  of  each  item  is  APPROPRIATE  for  the  TASK  LEVEL  and 
CONTENT  TYPF .  Use  the  table  below: 


CONTENT  TYPE 


TACK  IEVFL 

FACT 

CATEGORY 

PROCEDURE 

RULE 

PRINCIPLE 

REMEMBER 

for  RECOGNITION: 
matching 
true-false 
multiple  choice 
for  RECALL: 
short  answer 
fill-in 
listing 

short  answer 

fill-in 

listing 

short  answer 

fill-in 

listing 

short  answer 

fill-in 

listing 

short  answer 

fill-in 

listing 

USE-UNAIDED 

performance 
matching 
true-false 
multiple  choice 
short  answer 
fill-in 

performance 
t  ue-false 
multiple  choice 
short  answer 
fill-in 

performance 
true-false 
multiple  choice 
short  answer 
fill-in 

performance 
true-false 
multiple  choice 
short  answer 
fill-in 

USE-AIDED 

performance 
ma  tching 
true-false 
multiple  choice 
short  answer 

performance 
true-false 
multiple  choice 
short  answer 
fill-in 

performance 
true-false 
multiple  choice 
short  answer 
fill-in 

performance 
true-false 
multiple  choice 
short  answer 
fill-in 

STEP  1: 

STEP  ?: 

STEP  3: 


bTEP  4 

STFP  5 
STEP  6 


TEST  ADEQUACY 


Instructional 

Quality 

Inventory 


CARD  3 

DETERMINE  whether  or  not  each  item  is  CLEAR.  Instructions  for  completing  the  item  must  specify 
i.'h i f  ree;  'nee  the  student  is  expected  to  make. 

DETERMINE  whether  or  not  each  item  is  UNAMBIGUOUS.  Bach  item  must  have  one  and  only  one  correct 
>~c}orse,  -rid  ih .  item  crust  he  interpretable  in  only  one  way.  That  is,  items  must  not  he  confusing. 


DETERMINE  whether  or  not  each  item  is  WELL  CONSTRUCTED.  Different  c  rite  id  a  a^ply  to  different 

iter,  f 


IPUE-FALSE  items: 


MULTIPIE  CHOICE  items: 


MATCHING  items: 


FILL-IN  items: 


SHORT  ANSWER  items: 


LISTING  items: 


PERFORMANCE  items: 


An  item  should  include  only  one  statement  to  he 
judged  true  or  false. 

negative  statements  should  be  avoided. 

Don't  use  words  like  "never,"  "always,"  etc. 

Item  statements  should  be  short. 

All  alternatives  should  be  plausible. 

Negatives  in  the  item  stem  should  be  highlighted. 

Repetitive  phrases  should  be  placed  in  the  stem, 
not  in  the  alternatives. 

Alternatives  like  "all  of  the  above,"  "A  and  B  only," 
should  be  avoided. 

Instructions  should  explain  the  contents  of  each  column, 
and  explain  the  basis  for  matching. 

Instructions  should  specify  has  many  times  each  answer 
may  be. used. 

The  choice  column  should  include  extra  answers  unless 
answers  may  be  used  more  than  once. 

The  blank  should  be  at  or  near  the  end  of  the  sentence. 

One  and  only  one  phrase  should  correctly  complete  the  item. 

Multiple  blanks  should  be  avoided. 

Blanks  should  require  key  words. 

The  required  answer  should  be  kept  short. 

The  directions  to  the  student  should  specify  how  the 
item  will  be  scored. 

The  scoring  key  should  identify  allowable  synonyms  or 
alternatives. 

The  directions  should  specify  the  number  of  things  to 
be  listed  (if  appropriate  for  the  objective,  and  if 
the  number  of  things  is  not  a  hint). 

The  directions  should  specify  whether  or  not  order  is 
important.  If  so,  the  scoring  key  should  score 
senuen^s  separately. 

The  scoring  key  should  identify  allowable  synonyms  or 
alternatives,  and  should  specify  different  weights 
if  appropriate. 

The  directions  should  clearly  explain  what  the  student 
is  to  do  and  how  the  item  will  be  scored. 

The  scoring  key  must  svecify  all  criteria  the  performance 
must  meet,  such  as  completeness,  accuracy ,  quality, 
time  limit,  rate,  etc.  If  steps  in  the  performance 
are  scored,  a  checklist  should  be.  provided. 


DETERMINE  whether  or  not  each  item  is  FREE  of  HMTS.  An  item  should  not  give  away  the  answer 
to  i ten  If  or  to  any  other  item  on  the  test.  Tne  grammar  of  multiple-choice  and  fill-in  items 
should  not  give  hints  to  answers. 

DETERMINE  whether  or  not  the  items  allow  for  COMMON  ERRORS  to  be  made. 

For  USE-level  objectives,  OFTERMINE  whether  or  not  there  are  ENOUGH  ITEMS  to  test  the 
objective  adequately,  and  to  reflect  the  ranye  of  performance  required  on  the  job. 
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STEP  1:  This  step  refers  to  the  Test  Consistency  and  Adequacy  Form 
on  page  2  of  Volume  IV.  Again,  this  form  is  not  required; 
users  of  the  IQI  may  design  their  ohm.  form ,  or  use  no  form 
at  all. 


STEP  2:  The  Test  Consistency  procedure  should  be  applied  "objective 
by  objective. "  This  means  that  if  there  are  several  test 
items  for  a  single  objective 3  they  should  all  be  checked 
at  once.* 


STEP  3:  The  conditions  in  the  test  item,  or  the  conditions  under 

which  the  item  is  administered3  should  match  the  conditions 
specified  in  the  objective.  Naturally 3  there  are  some 
situations  when3  for  reasons  of  safety  or  practicality  or 
cost,  testing  conditions  cannot  be  exactly  the  same  as  the 
conditions  required  in  the  objective  or  on  the  job.  In  these 
cases3  it  is  important  to  simulate  the  conditions  as  closely 
as  possible.  Again 3  it  is  important  to  REMEMBER  THE  JOB ; 
the  testing  situation  must  be  close  enough  to  the  job  situation 
or  later  training  situation3  so  that  you  can  be  sure  that 
the  student  has  achieved  the  objectives. 


STEP  4:  The  standards  in  the  test  item ,  or  the  standards  for  scoring 
the  test  item3  must  match  the  standards  in  the  objective. 

In  criterion-referenced  testing 3  standards  are  not  arbitrarily 
selected.  It  makes  no  sense 3  for  example 3  to  require  a  student 
to  get  80%  of  the  items  right 3  if  he  needs  to  recall  all  the 
information.  On  the  other  hand ,  for  some  tasks3  a  70%  or  80% 
criterion  may  be  reasonable.  In  all  cases,  though 3  the  standard 
specified  in  the  objective  should  be  used. 


STEP  5:  The  task/content  level  of  the  test  item  should  match  the 
task/content  level  of  the  objective.  This  means  that  the 
action  verb  in  the  test  item  should,  be  the  same  as  the  action 
verb  in  the  objective,  or  at  least  the  same  behavior  must  be 
required.  If  it  isn't,  the  test  item  is  measuring  something 
different  than  was  required  in  the  objective. 

STEP  6:  The  steps  above  help  to  ensure  that  the  item  is  consistent  with 
the  objective.  Here,  we  want  to  ensure  that  the  item  is  re¬ 
presentative  of  the  kinds  of  situations  encountered  on  the  job, 
and  is  about  the  same  level  of  difficulty.  When  in  doubt, 
consult  a  subject-matter  expert. 

STEP  7:  There  are  a  number  of  different  test  item  formats,  and  these 
may  be  more  or  less  appropriate  depending  on  the  task/content 
level  of  ihe  objective.  The  chart  on  the  bottom  of  the  previous 
page  shows  the  acceptable  formats  for  each  task/content  level. 
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I ten  formats  at  the  REMEMBER  level 


In  the  chart  for  step  7,  notice  that  at  the  REMEMBER  level , 
recognition  items  ( multiple-choice ,  matching,  true- false)  are 
usually  not  appropriate.  This  is  because  they  don't  test 
recall ,  only  recognition.  Most  REMEMBER  level  objectives 
require  recall  because  of  the  nature  of  the  .job. 

Chapter  2  discussed  some  situations  in  which  recognition 
was  appropriate  for  REMEMBER  level  objectives.  For  these 
objectives ,  multiple-choice ,  matching,  or  true-false  test 
items  may  be  appropriate,  even  if  the  content  type  is 
category,  procedure,  rule,  or  principle.  These  do  not 
appear  on  the  chart  for  step  7,  because  even  though  they 
can  be  used,  they  are  not  the  best  choice.  In  this 
situation,  it  is  also  a  good  idea  to  recheck  a  recognition 
objective  to  make  sure  it  is  appropriate  for  the  job. 

Item  fonnats  at  the  USE  level 


Multiple-choice,  matching,  and  true- false  items  can  be 
appropriate,  if  carefully  designed,  for  many  VSE-level  tasks. 
For  example,  a  category  classification  is  often  a  true-false 
judgment.  If  the  student  must  solve  a  math  problem  (Use-Rule) , 
a  multiple- choice  item  in  which  all  alternatives  are  reasonable 
is  appropriate.  Also,  some  Use-Principle  predictions  involve 
a  limited  set  of  possible  alternatives ;  again,  multiple-choice 
is  appropriate. 

Why  is  format  important? 

The  reason  why  test  item  format  is  important  is  that  students 
are  not  dumb!  The  first  thing  most  new  students  do  in  a  course 
is  find  out  how  they  will  be  tested.  Then ,  they  study  just 
enough  to  pass  the  tests.  If  the  objective  requires  a  student 
to  memorize  something,  multiple-choice  tests  should  not  be 
used,  because  students  will  learn  just  enough  to  recognize, 
not  to  recall.  From  your  own  experience,  it  should  be  clear 
that  students  study  less  carefully  for  o.  multiple-choice  or 
true- false  test,  than  for  o.  completion  or  short-answer  test. 

The  test  items  and  the  format  should  be  like  the  tasks  the 
student  will  do  on  the  job. 


Additional  Explanation  for  the  TEST  ADEQUACY  procedure. 


The  information  on  the  previous  page  is  self-explanatory .  However, 
three  points  need  emphasis: 

First ,  instructions  to  the  students  are  important.  Students 
need  to  know  what  they  are  expected  to  do ,  and  how  they  will 
be  scored.  This  is  particularly  important  for  fill-in,  short- 
answer,  listing,  and  performance  test  items. 

Second,  scoring  criteria  should  be  very  carefully  specified, 
especially  for  short-answer  and  performance  items.  There 
should  be  explicit  criteria  or  steps  for  an  instructor  to 
use  for  scoring.  These  should  be  as  objective  as  possible, 
and  should  not  depend  on  individ-ial  judgment. 

Third,  steps  5  and  6  often  require  consultation  a 
subject  matter  expert.  That  person  can  tell  you  about  common 
errors,  and  can  usually  help  determine  whether  or  not  there 
are  enough  items. 

For  some  types  of  tasks,  there  are  methods  for  generati'ng 
test  items  to  cover  the  task  thoroughly.  Also,  there  are 
statistical  techniques  for  determining  if  enough  items  have 
been  given.  However,  these  methods  can  be  complicated  and 
costly. 
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EXAMPLES 


In  this  section,  we  have  repeated  the  "revised  objectives"  from 
chapter  2,  and  have  given  at  least  one  test  item  for  each.  For  the 
test  items,  we  will  first  determine  whether  or  not  the  item  is  consistent 
with  its  objective,  and  then  whether  or  not  the  item  is  adequate. 


1.  OBJECTIVE:  "The  student  will  state  the  rule  for  finding  total 
inductance  in  a  series  circuit." 

TEST  ITEM:  "Which  of  the  following  is  the  correct  rule  for  finding 
total  inductance  in  a  series  circuit? 


A.  E=IR  C.  R=IE 

B.  Xc=2lfRLc  D,  None  of  the  above. 


Consistent? 


Conditions  Match?  NO 


In  the  objective,  the  student  is  not  given  anything  except 
instructions  to  state  the  rule.  In  the  test  item,  however . 
the  student  is  given  several  rules,  one  of  which  might  he  the 
right  one. 

Standards  Match?  YES 


The  implied  standard  in  both  the  objective  and  the  test  item  is 
"100%  connect .  " 

Actions  Match?  NO 


The  task/content  classification  of  both  the  objective  and  test 
item  is  Remember-Rule.  The  action  in  the  objective,  however,  is 
"state”  while  the  action  in  the  test  item  is  really  "select"  or 
"choose. " 


Typical?  NO 

The  student  will  probably  never  have  to  choose  a  correct  formula 
from  a  bunch  of  alternatives  on  the  job.  The  reason  the  objective 
requires  recall  is  because  the  job  presumably  requires  recall. 
Therefore,  the  test  should  require  recall  too. 

Format  O.K.?  NO 


In  the  chart  for  step  7,  multiple- choice  is  not  a  good  format 
for  Remember-Rule.  It  is  obvious  why  the  format  is  bad ;  the 
multiple-choice  format  changes  the  conditions  and  action. 


Adequate? 


REVISED 
TEST  ITEM: 


Rot  Applicable.  If  the  item  is  not  consistent ,  there 
is  no  point  in  rating  adequacy.  An  inconsistent  item 
can  never  be  adequate.  At  this  point,  the  item  would 
be  rewritten  to  be  consistent. 

Even  if  the  item  were  consistent,  it  is  not  adequate 
because  " none  of  the  above"  is  not  a  good  alternative 
to  use  in  multiple-choice  questions. 

"In  the  space  below,  write  the  correct  rule  for  finding 
total  inductance  in  a  series  circuit." 
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2.  OBJECTIVE:  "Given  a  separate  picture  for  each  U.S.  Navy 

officer  uniform,  showing  collar  devices,  shoulder 
boards,  or  sleeve  stripes,  in  a  typical  job 
environment,  the  student  will  state  orally  from 
memory  the  rank  associated  with  each  collar  device, 
shoulder  board,  or  sleeve  stripe." 

TEST  ITEM:  "Column  A  below  shows  the  collar  devices  for 

different  ranks  of  naval  officers.  Column  B  gives 
different  ranks.  Match  each  item  in  column  A  with 
the  correct  rank  in  column  B." 


A.  1. 

B.  2.  (etc.) 


Consistent?  Conditions  Match?  NO 

The  item  should  give  pictures  of  entire  uniforms  in  typical  job 
situations ,  showing  shoulder  boards  or  sleeve  stripes  in  addition 
to  collar  devices.  Column  B  should  not  be  given  at  ally  because 
the  objective  does  not  say  that  the  student  is  given  the  ranks. 

Standards  Match?  YES 


Actions  Match?  NO 


Both  are  Remember-Fact, ,  but  the  objective  says  "state  orally" 
while  the  test  item  says  "match.  "  These  are  different  actions. 

Typical?  NO 

See  the  comr.ent  above  for  conditions. 

Format  O.K.?  NO _ 


In  the  chart  for  step  7,  matching  is  not  a  good  format  for 
Remember-fact  when  recall  is  required.  This  objective  requires 
recall. 


Adequate?  Rot  Applicable. 


REVISED  "These  pictures  show  Navy  officers  in  uniform.  Tell 

TEST  ITEM:  me  the  rank  of  the  officer  in  each  picture." 
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3.  OBJECTIVE:  “Given  any  resistor  with  four  color  bands,  the 

student  will  follow  the  procedure  for  determining 
ohmic  value  as  indicated  by  the  color  bands." 

TEST  ITEM:  "For  each  of  the  resistors  pictured  below,  determine 
the  value  in  ohms,  and  the  tolerance,  as  indicated 
by  the  colors." 


Consistent?  Conditions  Match?  ? _ 

The  objective  says  the  student  will  be  given  resistors ,  while 
the  test  item  gives  pictures.  This  might  be  close  enough  if 
the  pictures  are  life-size  and  in  color. 

Standards  Match?  YES 

Both  require  100%  accuracy. 

Actions  Match?  NO _ 

The  test  item  requires  the  student  to  determine  the  tolerance 
as  well  as  the  value.  Although  one  of  the  color  bands  does 
indicate  tolerance ,  this  was  not  required  in  the  objective. 

Typical?  ? _ 

See  the  conditions  question  above.  Another  question  concerns 
what  resistors  are  given  in  the  test  item.  Obviously ,  not  all 
resistors  can  be  tested.  Are  the  ones  which  are  tested  typical 
of  those  encountered  on  the  job,  or  are  they  "easy"  ones? 

Format  O.K.?  YES 


Adequate?  Not  Applicable. 


REVISED  "For  each  of  the  resistors  pictured  below,  determine 

TEST  ITEM:  the  value  in  ohms,  and  write  the  value  in  the  space 

next  to  each  resistor." 
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4.  OBJECTIVE:  "The  student  will  write  from  memory  the  steps  of  the 
procedure  for  message  reception  and  duplicate 
checking  as  listed  in  the  current  edition  of  NTP-4." 

TEST  ITEM:  "Use  the  current  edition  of  NTP-4  to  receive  the 
following  message  and  check  for  duplicates." 

Consistent?  Conditions  Match?  NO 

In  the  objectives  the  student  is  not  given  anything.  In  the  test 
item ,  the  student  is  given  a  message  and  is  given  NTP-4. 

Standards  Match?  YES 

Actions  Match?  NO 

The  objective  is  Remember-Procedure.  The  test  item  is  Use-Aided- 
Procedure. 

Typical?  NO 

The  Remember  objective  implies  that  there  is  a  later  Use-Unaided 
objective  requiring  the  student  to  perform  the  procedure  with  no 
memory  aid.  The  job}  there fore 3  must  require  the  same  performance 
with  no  memory  aid.  This  test  item  is  not  typical  because  it 
gives  the  memory  aid. 

Format  O.K.?  Not  Applicable. 

Adequate?  Not  Applicable. 

"In  the  space  below,  write  the  steps  of  procedure 
for  message  reception  and  duplicate  checking.  You 
will  receive  full  credit  if  you  list  all  steps 
exactly  as  stated  in  NTP-4  in  the  correct  order." 


REVISED 
TEST  ITEM: 


5.  OBJECTIVE:  "The  student  will  write  from  memory  the  principles 
of  operation  of  a  rotary  gear  pump,  as  described  in 
Tech.  Manual  XXX." 

TEST  ITEM:  "In  the  space  below  list  the  part  names  and 
functions  of  a  rotary  gear  pump." 

Consistent?  Conditions  Match?  YES 

The  implied  conditions  for  this  objective  are  " given  a  paper 
and  pencil . "  The  test  items  indicates  that  the  student  will 
have  a  paper  and  pencil  available. 

Standards  Match?  NO 

The  standards  for  the  objective  are  provided  in  Tech.  Nan.  XXX. 

It  is  not  clear  whether  this  manual  has  the  information  required 
by  the  test  item. 

Actions  Match?  NO 

The  task/ content  level  of  the  objective  is  remember-principle, 
while  the  task/content  level  of  the  test  item  is  remember- fact. 

Typical?  YES 

The  student  will  probably  have  to  know  the  part  names  and  functions 
of  a  rotary  gear  pump  in  order  to  operate  and  maintain  it. 

Format  OX?  YES 

Adequate?  Not  Applicable. 

REVISED  "In  the  space  below  write  the  principles  of  operation 

TEST  ITEM:  of  a  rotary  gear  pump,  as  they  were  described  in 

Tech.  Manual  XXX." 
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6,  OBJECTIVE:  "Given  a  globe  valve,  rags  Prussian  blue,  gasket 
material,  packing,  tools,  and  an  MRC  job  program 
card,  the  student  will  disassemble  and  reassemble 
the  globe  valve  (within  30  minutes)." 

TEST  ITEM:  "Which  of  the  following  materials  are  required  for 
disassembling  and  reassembling  the  globe  valve?" 

A.  rags  C.  gasket  material 

B.  Prussian  blue  D.  All  of  the  above. 

Consistent?  Conditions  Match?  NO 
Standards  Match?  NO 


Actions  Match?  NO 


Typical?  _ NO 

Format  O.K.?  NO 


It  should  be  obvious  that  this  test  item  does  not  address  the 
objective  at  all. 


Adequate?  Not  Applicable. 


REVISED  "Disassemble  and  reassemble  your  globe  valve 

TEST  ITEM:  using  the  MRC  card.  Tools  and  materials  are 

on  the  workbench.  You  have  30  minutes." 
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7.  OBJECTIVE:  "The  student  will  classify  any  call  sign  according 
to  one  of  the  following  types  --  international  U  S. 

Navy  ship,  international  U.S.  Navy  shore,  indefinite, 
task  organization,  voice,  or  not  a  valid  Navy  cail  < i gn . 
Type  names  will  not  be  provided  during  classificat .on. 

TEST  ITEM:  "For  each  of  the  call  signs  listed  below,  write  the 
type  in  the  space  provided. 

NTQT  _ 

T2RA  _ _ 

(etc.) 

Consistent?  Conditions  Match?  YES 
Standards  Match?  YES 

Actions  Match?  YES _ 

Typical?  ? _ 

We  will  assume  that  the  call  signs  to  he  classified  are  typical 
of  those  the  student  will  see  on  the  job. 

Format  O.K.?  YES 


Adequate?  Clear?  YES 

Unambiguous?  YES 

Well  Constructed?  ? _ 

The  overall  test  directions  should  specify  how  the  items  will 
he  scored.  The  scoring  key  should  specify  to  the  instructor 
how  to  score  the  items. 

Free  o'  Hints?  YES 

Common  Errors?  ? _ 

Enough  Items?  ? _ 

The  items  should  he  reviewed  by  a  subject-matter  expert  to  determine 
if  there  are  common  errors  which  should  be  tested ,  and  to  determine 
whether  or  not  there  are  enough  items  so  that  we  are  sure  the 
student  will  he  able  to  perform  well  on  the  job. 

REVISED  This  test  item  is  probably  O.K. ,  assuming  that  test 

TEST  ITEM:  directions  and  scoring  are  clear ,  and  that  common 

errors  and  enough  items  are  present. 
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8.  OBJECTIVE:  "Given  the  formula  for  Ohm's  Lav/,  E=IR,  and  any  two 
values  typical  of  the  values  encountered  on  the  job, 
the  student  will  solve  for  the  third  value  correct 
to  two  decimal  places." 

TEST  ITEM:  "Ohm's  Law  is  E=IR.  If  1=200  ma.,  and  R=47K  ohms, 
then  E= _ . 

A  C 

B....  D.  .  .  . 

Consistent?  Conditions  Match?  YES 
Standards  Match?  ? 


The  item  should  specify  that  accuracy  to  too  decimal  places  is 
required. 


Actions  Match?  YES 

The  student  has  to  solve  the  problem  to  answer  the  test  item. 

Typical?  ? _ 

We  will  assume  the  numbers  given  are  typical  of  the  job. 


Format  O.K.?  YES 

Adequate?  Clear?  YES 

Unambiguous?  YES 

Well  Constructed?  ? 


We  will  assume  that  all  the  alternatives  are  plausible. 

Free  of  Hints?  ? 


This  item  gives  Ohm's  Law.  There  had  better  not  be  another  item 
on  the  test  which  asks  the  student  to  recall  Ohm's  Law. 

Common  Errors?  ? 


The  alter 7,  ;  ves  should  be  designed  so  that  they  include 
answers  that  the  student  might  choose  if  he  did  the  problem  wrong. 

Enough  Items?  ? _ 

There  should  be  enough  items  of  this  type  so  that  we  are  sure 
the  student  can  solve  any  new  Ohm's  Law  problem  he  might  see. 

REVISED  The  item  is  probably  O.K. 

TEST  ITEM: 
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9.  OBJECTIVE:  "The  student  will  recall  in  the  proper  order 

the  procedures  performed  by  a  gun  captain  while 
firing  during  general  quarters." 

TEST  ITEM:  "List  the  duties  performed  by  a  gun  captain." 

Consistent?  Conditions  Match?  YES _ 

Standards  Match?  YES _ 

Actions  Match?  _ YES _ 

Typical?  ? _ 

It  is  assumed  that  the  student  needs  to  know  this  to  do  the  job. 

Format  O.K.?  YES _ 

Adequate?  Clear?  NO _ 

The  test  item  should  specify  what  duties. 

Unambiguous?  NO _ 

If  gun  captains  perform  various  duties  under  different  conditions t 
then  there  could  be  more  than  one  correct  answer  to  this  item. 


REVISED 
TEST  ITEM: 


Well  Constructed? 
Free  of  Hints? 
Common  Errors? 
Enough  Items? 


YES _ 

YES _ 

Not  Applicable 
Not  Applicable 


"In  the  space  below  write  in  the  proper  order  the 
procedures  .  gun  captain  performs  while  firing 
during  general  quarters." 
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10.  OBJECTIVE:  "In  a  classroom  exercise  simulating  shipboard 
working  conditions,  the  student  will  file  drill 
messages  identified  as  Top  Secret,  ~*cret,  special 
category,  readdressed  general  mess<.djs,  and  other 
classified  and  unclassified  messages,  in  the 
correct  files,  at  the  rate  of  one  message  every 
60  seconds." 

TEST  ITEM:  (Instructor)  "O.K. ,  guys,  the  next  drill  is 
message  filing.  You  will  file  the  next  thirty 
messages  you  get." 


Consistent?  Conditions  Match?  ? _ 

We  will  assume  that  this  drill  will  be  conducted  as  a  classroom 
exercise  simulating  shipboard  conditions ,  and  that  all  the 
different  messages  in  the  objective  are  part  of  the  test. 

Standards  Match?  ? 


The  standard  is  not  given  in  the  item ,  but  could  be  part 
of  the  scoring  system. 

Actions  Match?  YES 


Typical?  YES 

If  the  dri.ll  simulates  shipboard  conditions ,  then  it  is  typical 
of  the  job. 

Format  O.K.?  YES 


Adequate?  Clear?  YES 

Unambiguous?  YES 

Well  Constructed?  NO 


The  student  is  told  what  to  do,  but  he  is  not  told  how  he  will  be 
scored.  He  should  be  told  the  "60  sec.  per  message"  standard,  and 
the  accuracy  requirements. 

Free  of  Hints?  YES 

Common  Errors?  ? _ 

Enough  Items?  ? _ 


Message  types  that  are  commonly  misfiled  should  be  included  in  the 
exercise.  Thirty  messages  is  probably  enough;  ask  a  subject-matter 
expert. 


REVISED  O.K. ,  except  add  the  standards:  "You  have  SO  minutes , 

TEST  ITEM:  and  all  messages  must  be  in  the  correct  files." 
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11.  OBJECTIVE:  "Given  schematic  symbols  for  common  electronic 
components,  the  student  will  write  from  memory 
the  name  of  the  component  represented  by  each 
symbol . " 

TEST  ITEM;  " _ is  the  symbol  for  resistor." 


Consistent?  Conditions  Match?  NO 

The  objective  says  that  the  student  is  given  syrribols.  The  test 
item  does  not  give  a  symbol. 

Standards  Match?  YES 

Actions  Match?  NO _ 

The  objective  says  " write  the  name. "  The  test  item  asks  the 
student  to  draw  the  symbol. 

Typical?  ? _ 

Does  the  student  have  to  draw  schematic  symbols  on  the  job? 
Format  O.K.?  YES 


Adequate?  Dot  Applicable. 


REVISED  "Several  schematic  symbols  for  common  electronic 

TEST  ITEM:  components  are  shown  below.  Write  the  name  of 

the  component  represented  by  each  symbol  in  the 
space  provided," 
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12.  OBJECTIVE:  "Given  the  explanation  of  the  principle  of 

supply  of  a  military  force  from  the  text,  the  student 
will  discuss  how  this  principle  applies  in  Hitler's 
attack  on  Russia,  the  Battle  of  Midway,  the  Battle 
of  the  Bulge,  and  Sherman's  march  through  Georgia. 

This  discussion  should  include  the  following  points.  .  ." 

TEST  ITEM:  "Using  the  explanation  of  the  principle  of  supply 
of  a  military  force  given  in  your  book,  discuss  how 
these  principles  apply  in  Hitler's  attack  on 
Russia,  the  Battle  of  Midway,  the  Battle  of  the 
Bulge,  and  Sherman's  march  through  Georgia." 

Consistent?  Conditions  Match?  VES 

Standards  Match?  V _ 

The  scoring  standards  would  include  the  discussion  points  given 
in  the  objective. 


Actions  Match? 

YES 

Typical? 

YES 

Format  O.K.? 

YES 

Adequate?  Clear? 

Unambiguous 
Well  Constructed? 

There  are  probably  many  ways  that  students  could  answer  this  question. 
The  scoring  standards  must  be  as  objective  as  possible  in  specifying 
how  students '  responses  will  be  scored.  Also ,  more  instructions  to 
the  student  about  what  his  answer  should  contain  would  help  reduce 
the  variability  of  answers.  More  instructions,  however,  might  also 
be  too  much  of  a  hint. 

Free  of  Hints?  ? _ 

The  book  that  gives  the  principle  of  supply  had  better  not  give 
away  any  answers  to  this  question. 

Common  Errors  ? _ 

Enough  Items?  ? _ 

Ask  a  subject-matter  expert. 


REVISED  Add:  "You  will  receive  full  credit  if  your  discussion 

TEST  ITEM:  includes  the  following  points . " 


13.  OBJECTIVE:  "The  student  will  recall  in  writing  the  1 of a r gram 

characteristics  for  the  following  types  of  contacts: 
merchant  ship,  aircraft  carrier,  destroyer,  whale, 
diesel  submarine,  nuclear  submarine." 

TEST  ITEM:  "In  the  spaces  below,  write  the  lofargram  characteristics 
you  would  expect  for  each  of  the  types  of  contacts. 

Merchant  ship:  _ _ 


Consistent?  Conditions  Match?  YES 

Standards  Match?  YES 

Actions  Match?  YES 

Typical?  YES 

Format  O.K.?  YES 

Adequate?  Clear?  YES 

Unambiguous?  YES 


Well  Constructed?  ? _ 

The  instructions  should  tell  the  student  how  the  answers  will  be 
scored.  Will  -partial  credit  be  given  for  partial  answers? 

Free  of  Hints?  YES 


Common  Errors?  YES 


Enough  Items?  Wot  Applicable 


REVISED 
TEST  ITEM: 


Add  instructions  which  specify  how  the  answers  will 
be  scored. 
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14.  OBJECTIVE:  "Given  a  circuit  schematic  with  values  of 
frequency  and  inductance  specified,  the 
student  will  solve  for  inductive  reactance." 

TEST  ITEM:  "In  the  circuit  below,  the  frequency  and  inductance 
are  given.  Calculate  the  inductive  reactance." 


Consistent?  Conditions  Match?  YES 
Standards  Match?  YES 

Actions  Match?  YES 

Typical?  ? 


We  will  assume  that  the  circuit 3  and  the  values  of  frequency 
and  inductancej  are  typical  of  those  encountered  on  the  fob. 


Format  O.K.?  YES 

Adequate?  Clear?  YES 

Unambiguous?  YES 

Well  Constructed?  YES 

Free  of  Hints?  YES 

Common  Errors?  ? 

Enough  Items?  ? 


Check  with  a  subject-matter  expert  to  determine  if  common  errors 
have  been  tested 3  and  if  enough  items  have  been  given. 


REVISED 
TEST  ITEM: 


O.K. 


15.  OBJECTIVE:  "In  a  classroom  laboratory  simulating  a  ship 
radio  room,  the  student  will  perform  the 
steps  required  to  process  (accept,  verify, 
log)  messages  to  be  transmitted  via  teletype 
tape,  at  the  rate  of  5  minutes  per  message." 

TEST  ITEM:  "The  following  message  will  be  transmitted  via 
teletype  tape.  Accept  it,  verify  it,  and  log 
it.  You  will  receive  full  credit  if  you  perform 
these  steps  accurately  within  30  seconds." 


Consistent?  Conditions  Match? _ ? _ 

We  will  assume  that  the  student  is  in  the  classroom  laboratory. 
Standards  Match?  NO 

The  objective  says  5  minutes  per  message.  The  test  item  says 
30  seconds. 

Actions  Match?  YES 

Typical?  ? _ 

The  test  item  is  typical  to  the  extent  that  actual  job  situations 
are  accurately  simulated. 

Format  O.K.?  YES 


Adequate?  Clear?  YES _ 

Unambiguous?  YES 

Well  Constructed?  YES 

Free  of  Hints?  ? _ 

Do  the  words  " accept } "  "verify 3  "  and  "log"  give  away  the  procedure 
,  to  be  followed? 

Common  Errors?  ? _ 

Enough  Items?  ? _ 

Again 3  a  subject-matter  expert  should  be  consulted. 


REVISED  "The  following  message  will  be  transmitted  via 

TEST  ITEM:  teletype  tape.  Process  it  according  to  the 

procedure  you  have  learned.  You  will  receive 
full  credit  if  you  perform  the  steps  accurately 
within  5  minutes." 
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16.  OBJECTIVE:  "The  student  will  state  the  principles  of 

operation  of  a  jet  pump  as  described  in  Technical 
Manual  A-xxx." 

TEST  ITEM:  "List  the  names  of  the  parts  of  a  jet  pump." 

Consistent?  Conditions  Match?  YES _ 

Standards  Match?  NO _ 

The  standards  for  the  objective  are  provided  in  Tech.  Man.  A-xxx. 

It  is  not  clear  whether  this  manual  has  the  information  required 
by  the  test  item. 

Actions  Match?  NO _ 

The  task/content  le”el  of  the  objective  is  remember-principle } 
while  the  task/content  level  of  the  test  item  is  remember- fact. 

Typical?  YES _ 

Student  will  probably  have  to  know  the  parts  of  a  jet  pump. 

Format  O.K.?  YES _ 

Adequate?  Not  Applicable. 

REVISED  "In  the  space  below, write  the  principles  of  operation 

TEST  ITEM:  of  a  jet  pump  as  they  were  described  in  Tech.  Man.  A-xxx." 


17.  OBJECTIVE:  "The  student  will  use  the  principle  of  electro¬ 
magnetic  induction  to  describe  the  operation  of 
an  AC  generator.  The  description  must  contain 
the  following  points...." 

TEST  ITEM:  "In  the  space  below,  describe  how  an  AC  generator 
works." 


Consistent?  Conditions  Match?  YES 

Standards  Match?  ? _ 

The  scoring  standards  for  the  item  must  match  the  objective. 

Actions  Match?  _ YES _ 

Typical?  ? 

On  the  job}  will  the  student  have  to  give  new  explanations 
of  the  operation  of  equipment  based  on  his  knowledge  of 
theory.  Does  the  knowledge  of  the  principle  of  electro¬ 
magnetic  induction  contribute  to  better  job  performance? 

Format  O.K.?  YES 


Adequate?  Clear? 

Unambiguous? 

Well  Constructed? 

Is  it  clear  what  response  the  student  is  expected  to  give? 

The  instructions  to  the  student  should  explain  exactly  what 
the  student  should  do  to  answer  the  question ,  and  should 
explain  how  the  answer  will  be  scored.  The  scoring  key 
should  also  specify  how  the  "points"  mentioned  in  the  objective 
will  be  assessed. 

Free  of  Hints?  YES 

Common  Errors?  ? _ 

Enough  Items?  _ ? _ 

This  is  a  Use-unaided  Principle.  Therefore 5  we  expect  that  the 
studenr,  will  have  to  use  the  principle  to  explain  how  new  or  '■un¬ 
familiar  equipment  operates.  There  must  be  enough  items  so  that 
we  one  sure  the  student  can  apply  his  knowledge  of  electromagnetic 
induction  in  job  situations. 

REVISED  "In  the  space  below,  describe  in  terms  of  the  principle 

TEST  ITEM:  of  electromagnetic  induction  how  an  AC  generator 

works.  Your  answer  should  include  the  following  points 
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18.  OBJECTIVE:  "Given  the  guidelines  for  determining  message 
security  classification,  the  student  will 
determine  the  security  classification  (Top  Secret, 
Secret,  Confidential,  or  Unclassified)  for  outgoing 
messages." 

TEST  ITEM:  "List  the  four  types  of  security  classifications." 


Consistent? 

Conditions  Match? 

NO 

Standards  Match? 

YES 

Actions  Match? 

NO 

The  conditions  and  action  in  the  objective  are  different  from 
those  in  the  test  item.  The  task/content  level  of  the  objective 
is  Use- Aided  Category }  while  the  task/content  level  of  the  test 
item  is  Remember- Fact. 

Typical?  _ ? _ 

The  job  probably  requires  the  student  to  know  the  four  types 
of  security  classifications.  This  information  should  have 
been  covered  in  a  previous  objective. 

Format  O.K.?  YES 


Adequate?  Not  Applicable. 


REVISED  "Several  sample  outgoing  messages  are  given  below. 

TEST  ITEM:  For  each  message,  use  the  guidelines  for  message 

security  classification  to  determine  the  appropriate 
security  classification." 
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Chapter  4 

PRESENTATION  CONSISTENCY 


Introduction 


At  this  point,  the  objectives  and  test  items  for  the  instructional 
program  are  consistent  with  each  other,  and  are  adequate.  The  next  step 
is  to  make  sure  that  the  instructional  presentation  is  consistent  with 
the  objectives  and  test  items.  - - 

The  term  "instructional  presentation"  covers  all  the  ways  instruction 
can  be  presented.  This  includes  printed  self-study  materials,  lectures, 
computer-assisted  instruction,  films  or  videotapes,  tape-slide  presentations, 
audio  tape,  videodisc,  one-on-one  tutorials,  and  any  combination  of  these. 

For  a  presentation  to  be  consistent  with  an  objective/test  item, 
it  must  teach  to  the  task  level  and  content  type  of  the  objective/test 
item.  This  means  that  the  presentation  must  contain  certain  components, 
depending  on  the  task/content  level.  There  are  four  main  presentation" 
components:  - 

The  student  is  given  a  statement  of  a  fact,  a 
category  definition,  the  steps  of  a  procedure 
or  rule,  or  a  statement  of  a  principle. 

The  student  is  told  or  shown  how  a  statement  of 
a  category,  procedure,  rule,  or  principle  applies 
in  a  specific  case. 

The  student  is  asked  to  supply  part  or  all  of  a 
fact  statement,  category  definition,  the  steps  of 
a  procedure  or  rule,  or  a  statement  of  a  principle. 

The  student  is  also  given  feedback  about  the 

correctness  of  his  answer. 

The  student  is  asked  to  use  a  category  definition, 
procedure,  rule,  or  principle  on  a  specific  case 
to  which  it  applies,  and  is  given  feedback  about 
the  quality  of  his  performance. 

Different  combinations  of  these  components  are  required,  depending  on  the 
task  level  of  the  objective. 

A  component  is  counted  as  present  in  the  instruction  only  if  it  is 
complete.  This  depends  on  the  content  type  of  the  objective,  and  simply 

means  that  everything  that,  needs  to  be  included  is  included. 


Statement 


Example 


Practi ce 
Rememberi ng 


Practice 

Using 


On  the  following  page,  the  PRESENTATION  CONSISTENCY  procedure  from 
Volume  IV  is  reproduced.  Then,  additional  explanation  is  given  and 
example  are  provided. 
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PRESENTATION  CONSISTENCY 


Instructional 

Quality 

Inventory 


STE°  1:  ENTER  the  COURSE  TITLE  and  OBJECTIVE  NUMBER  at  the  top  of  the  fora. 

STEP  2:  LOCATE  the  section  of  the  presentation  related  to  this  objective. 


STEP  3:  DETERMINE  if  the  required  PRESENTATION  COMPONENTS  are  present  for  the  task  level 
oTtKe  wDjective.  (Use  the  table  below). 


FKCVIFSD  FBESKSTATIiV;  CSypQXWT.h 


T*'K  ism 

STATEMENT 

PRACTICE 

REMEMBERING 

EXAMPLES 

PRACTICE 

USING 

REMEMBER 

required 

required 

not 

required 

not 

required 

USE-UNAIDED 

retired  w 
aeeccic.ted 
objective  y 
recently 

lees  the 
F£?!E!-!BEF 

28  tauiht 

required 

required 

USE-AIDED 

thy  nid 

replacuS 

statement 

net 

required 

required 
uHth  at! 

required 
vith  aid 

ST"?  4:  DETERMINE  whether  or  not  each  required  PRESENTATION  COMPONENT  is  COMPLETE  for  the  content 
type  of  the  objective  (Use  the  table  below).  Examples  and  pract  ce  items  must  also  match 
the  task  level  of  the  objective. 


PRESENTATION 

COMPONENT 

CONTENT  TYPE  OF  THE  OBJECTIVE 

FACT 

CATEGORY 

PROCEDURE 

RULE 

PRINCIPLE 

STATEMENT 

complete  fact 
presen  ,ed 

all  critical 
characteristics 
and  their 
combination 
are  given 

all  steps  are 
given  in  the 
correct  order 

all  steps  and 
branching 
decisions  are 
given  in  the 
correct  order 

all  causes, 
effects,  and 
relationships 
are  given 

PRACTICE 

REMEMBERING 

recall  or  recog¬ 
nition  required 

recall  of 
category  def¬ 
inition  required 

recall  of  all 
steps  in  correct 
order  required 

recall  of  all 
steps  and  branch 
decisions  in 
correct  order 
required 

recall  of  «11 
causes,  effects, 
relationships 
required 

For  all  content  types:  Practice  Remembering  items  must  be  the  same  as  the 
They  must  be  the  same  format  as  the  test  item. 

All  practice  items  must  include  feedback. 

test  item. 

EXAMPLES 

not 

applicable 

examples  show 
all  critical 
characteristics 
required  for 
classification, 
non-examples  show 
absence  of 
critical 
characteristics 

application  of 
the  procedure 
must  be  shown 
and  steps  must 
be  shown  in  the 
correct  order 

application  of 
each  step  or 
branching 
decision  must 
be  shown  in  the 
correct  order 

interpretation 
or  prediction 
based  on  causes, 
effects,  and 
relationships 
must  be  shown 

PRACTICE 

USING 

not 

applicable 

classification 
of  both  examples 
and  non-examples 
is  required 

all  steps  must 
be  performed 
in  the  correct 
order 

all  steps  and 
branching 
decisions  must 
be  performed  in 
the  correct  order 

explanation  or 
prediction 
based  on  the 
principle  is 
required 

For  all  content  types:  Practice  Using  items  must  reflect  what  is  to  be  done  on  the 
job  or  in  later  training. 

The  task/content  level,  conditions,  and  standards  must  match 
the  test  item  and  objective. 

The  practice  item  format,  must  be  the  same  as  the  test  item  format. 
All  practice  items  must  include  feedback. 

Some  practice  items  should  be  different  than  either  the  test 
items  or  the  examples.  (Cannon  error  itemb  might  be  the  same.) 
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For  CATEGORIES, 
RULES,  PRINCIPLES: 


Additional  Explanation  for  the  PRESENTATION  CONSISTENCY  prose  Jure. 


STEP  1:  This  step  refers  to  the  Presentation  Consistency  and  Adequacy 
Form  on  page  3  of  Volume  IV.  Again ,  this  form  is  not  required. 

STEP  2:  Presentation  consistency  should  be  done  objective-by-objective, 

or  on  groups  of  related  objectives .  In  either  case ,  the  instruc¬ 
tion  should  be  reviewed  to  locate  the  section(s)  related  to  the 
objective (s) .  In  printed  materials ,  the  components  may  riot 
always  be  in  the  same  place.  For  example,  practi.ee  may  be  in 
a  separate  workbook,  or  at  the  end  of  a  chapter.  For  lectures, 
films,  etc.,  the  entire  presentation  may  have  to  be  reviewed. 
However,  instructor  guides  or  lecture  notes  or  film  scripts 
may  help  in  locating  the  appropriate  section. 

STEP  3:  The  table  on  the  previous  page  shows  which  presentation  components 
are  required  for  each  task  level.  These  required  components  apply 
across  all  content  types  for  the  REMEMBER  level,  and  all  except 
facts  for  the  USE  level. 

The  table  is  mostly  self-explanatory.  However,  notice  the  USE- 
UNAIDED  level.  Step  4f  of  the  Objectives  Adequacy  procedure 
requires  that  there  be  a  previous  Remember  objective  for  a  Use- 
unaided  objective.  If  this  previous  Remember  objective  is 
covered  in  the  some  presentation,  then  a  statement  and  some 
practice  remembering  will  be  required.  Therefore,  it  is  not 
necessary  to  repeat  the  statement  and  the  practice  remembering. 

STEP  4:  A  presentation  component  is  not  present  unless  it  is  complete. 

Some  completeness  guidelines  are  shown  in  tine  table,  and  are 
generally  self-explanatory.  However ,  the  role  of  practice 
questions  in  both  REMEMBER  and  USE  level  instruction  needs 
more  explanation. 

Practice  questions  or  exercises  are  included  in  instruction 
because  they  (1)  focus  on,  and  call  attention  to,  infor¬ 
mation  or  tasks  that  must  be  learned,  (2)  inform  the 
student  about  what  the  lest  will  be  like,  and  (3)  allow 
the  student  to  monitor  his  own  learning. 

For  these  reasons.  Practice  Remembering  items  must  (1)  test  the 
same  information  as  the  test,  (2)  be  the  same  format  as  the  test 
so  the  student  is  not  "tricked,"  and  (3)  must  have  feedback. 

For  the  same  reasons.  Practice  Using  items  or  exercises  must 

(1)  reflect  what  is  to  be  done  on  the  job ,  and  match  the  task/ 
content  level,  conditions ,  and  standards  in  the  objective, 

(2)  be  the  same  format  as  the  test,  and  (3)  include  feedback  so 
the  student  can  monitor  his  learning.  It  is  important  to  em¬ 
phasise  that  Practice  Using  items  for  concepts,  rules,  and 
principles  must  be  different  than  the  test  items  and  examples 
(even  though  they're  the  same  format).  They  must  be  different 
because  these  content  types  require  the  student  to  be  able  to 
deal  with  problems  or  situations  that  are  new.  (For  procedures , 
nothing  new  is  required,  so  the  practice  is  the  same  as  the  test. ) 
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There  are  two  exceptions  to  the  rules  above.  Firsts  practice 
is  sometimes  graded  in  difficulty ,  because  the  final  task  is 
too  difficult  for  a  beginner  to  perform.  In  this  situation , 
early  practice  items  are  simplified ,  and  are  therefore  not 
identical  with  job  performance. 

The  second  exception  can  arise  when  a  decision  has  been  made 
to  use  an  inappropriate  test  item  format  for  time  and  cost 
reasons.  For  example ,  the  multiple-choice  format  is  often 
used  to  test  at  the  remember  level,  even  though  it  is  not 
appropriate.  In  this  situation ,  the  practice  i tetri  format 
should  be  the  appropriate  format ,  not  the  inappropriate  format 
of  the  test  item. 


The  best  way  to  do  s'eps  3  and  4  above  is  to: 

a.  Determine  the  task/content  level  of  the  objective. 

b.  Use  the  task  level  of  the  objective  and  the  chart  for 
step  3  to  determine  what  presentation  components  must 
be  present.  Write  them  down. 

c.  Use  the  content  type  of  the  objective  and  the  chart  for 
step  4  to  determine  the  compleveness  requirements  for  each 
component  you  have  written  down.  Write  these  down  too, 

in  terms  of  the  subject-matter  topic  you  are  dealing  with. 

d.  Using  what  you've  written  down  as  a  guide,  go  through  the 
presentation  to  check  it  for  consistency. 

e.  Save  your  written  notes,  and  the  locations  of  the  presentation 
components ,  for  the  Objectives  Adequacy  procedure  (in  the  next 
chapter) . 
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EXAMPLES 


In  this  section,  we  will  present  three  examples  of  instructional 
presentations,  along  with  their  associated  objectives  and  test  items. 
The  first  page  of  each  example  will  give  the  objective  and  test  item, 
and  the  analysis  procedure  for  consistency  as  described  on  the  previous 
page.  On  the  next  page  or  pages,  the  instruction  will  be  presented 
and  consistency  will  be  evaluated.  Our  comments  about  cons is ter  v 
will  be  given  in  italics  in  the  margins. 

In  the  next  chapter,  the  same  examples  will  be  repeated  and 
checked  for  adequacy.  Then,  revised  versions  of  the  examples,  which 
will  be  consistent  and  adequate,  will  be  given. 
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Example  1 . 
OBJECTIVE: 


TEST  ITEM: 


Task/Content 

Classification: 


"Given  a  line  drawing  of  a  basic  reciprocating 
pump,  the  student  will  write  the  correct  part 
names  in  the  spaces  provided.  Parts  to  be 
labeled  are:  cylinder,  piston,  valve,  connecting 
rod,  handle." 

"A  line  drawing  of  a  basic  reciprocating  pump 
is  shown  below.  Label  the  parts  in  the  spaces 
provided." 


Remember  -  Fact 


Required 

Presentation 

Components: 


Statement  of  the  facts  to  be  recalled  (names 
of  the  pump  parts,  and  their  locations 
on  the  line  drawing). 

Practice  Remembering  the  facts  (names  of  the 
pump  parts,  and  their  locations). 


Completeness  Statement:  All  part  names  in  the  objective 

Requirements:  should  be  given,  and  their  locations  on 

the  line  drawing  should  be  indicated. 

Practi ce  Rememberi ng :  Recall  cf  the  correct 
part  names,  as  in  the  test  item,  is 
required.  Feedback  must  be  present,  or 
else  this  component  is  not  complete. 


The  instructional  presentation  is  given  o'-  the  next  page. 

We  will  try  to  find  the  required  presentation  components 3  and  then 
determine  whether  or  not  they  are  complete. 
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Instruction: 


Pumps 

Pumps  provide  the  force  required  to 
move  fluids.  They  do  this  by  establishing 
and  maintaining  a  difference  in  pressure 
between  the  inlet  and  (discharge)  outlet 
of  the  pump.  Did  you  know  that  you  carry 
a  pump  around  with  you?  All  the  time? 

Just  like  your  own  heart,  the  "heart"  of 
most  fluid  transfer  systems  is  a  pump. 

Aboard  ship,  pumps  are  used  to  move 
water,  oil,  fuel,  and  air.  These  fluids 
and  gases  are  moved  by  pumps  into  and  out 
of  storage  tanks,  piping  systems,  and 
machinery  to  move  the  ship,  make  elec¬ 
tricity,  and  keep  the  crew  safe  and 
comfortable.  Without  its  pumps,  a  steam 
or  diesel-powered  ship  couldn't  even  get 
underway . 

There  are  many  different  kinds  of 
pumps.  A  basic  reciprocating  pump  is  a 
pump  which  is  used  to  push  air  into  various 
receptacles  such  as  a  basketball  or  a 
bicycle  tire.  This  is  accomplished  by 
having  the  connecting  rod -(which  connects 
the  piston  to  an  outside  energy  source) 
push  the  piston  (plunger-shaped  mechanism) 
down,  forcing  air  out  of  the  cylinder 
(which  acts  as  the  encasement  for  the 
piston  and  air)  and  through  a  valve. 

The  purpose  of  the  valve  in  a  basic 
reciprocating  pump  is  to  enable  gas 
(e.g.,  air)  to  enter  and  escape  from 
the  cylinder. 

Presented  below  is  a  diagram  which 
illustrates  the  major  parte  of  basic 
reciprocating  pumps. 

^CYLINDER 


PISTON, 


^4-VALVE 


-HANDLE 


CONNECTING 

ROD 


The  STATEMENT  is 
given  in  the  picture 
at  the  bottom,  and 
in  paragraph  3, 


There  is  no  PRACTICE 
REMEMBERING. 
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Example  2. 

OBJECTIVE:  "The  student  will  sharpen  a  pocket  knife 

using  prescribed  procedures,  given  oil,  a 
sharpening  stone,  and  a  pocket  knife." 

TEST  ITEM:  "Sharpen  your  pocket  knife.  Your  instructor 

will  watch  and  evaluate  your  performance." 

Task/Content 

Classification:  Use-unaided  -  Procedure 

Since  this  objective  is  Use-unaided ,  we  have  included  a  previous 
Remember  objective,  as  required  in  step  4f  of  the  Objectives 
Adequacy  procedure . 

Previous  Remember  "The  student  will  write  from  memory  the  steps 

Objective:  of  procedure  for  sharpening  a  pocket  knife." 

Test  Item:  "List  the  steps  of  procedure  for  sharpening 

a  pocket  knife  in  the  space  below." 

In  this  simple  situation,  the  previous  Remember  objective  (and  test 
item)  could  possibly  be  omitted,  because  if  the  student  can  do  the 
task,  we  can  infer  that  he  remembers.  In  situations  that  are  complicated, 
or  involve  safety  or  expensive  equipment,  previous  Remember  objectives 
must  be  included. 


Required  Statement  of  the  steps  of  procedure  for 

Presentation  sharpening  the  pocket  knife. 

Components: 

Practice  Remembering  the  steps  of  procedure. 

Example  showing  the  procedure  being  used. 

Practice  Using  the  sharpening  procedure.  The 
student  practices  sharpening  according  to 
the  procedure. 


Completeness  Statement:  All  steps  must  be  given  in  the 

Requirements:  correct  order. 

Practice  Remembering:  This  must  be  like  the 
test  item  for  the  Remember  objective. 
Feedback  must  be  given. 

Examples:  Each  step  of  the  procedure  should 
be  illustrated  or  demonstrated. 

Practice  Using:  The  student  should  practice 
sharpening  his  knife.  The  instructor 
should  supervise,  and  must  provide  feedback. 
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Instruction: 


Sharpening  a  pocket  knife 
on  a  sharpening  stone: 

Most  pocket  knives  may  be 
sharpened  on  a  medium  or  fine 
grade  sharpening  stone  with  a 
few  drops  of  oil  spread  on  the 
surface.  Hold  the  handle  of  the 
knife  in  one  hand,  and  place  the 
blade  across  the  stone.  Press  the 
blade  down  with  the  fingers  of  the 
other  hand,  and  stroke  the  blade 
in  a  circular  motion  as  shown  below. 
After  several  strokes,  reverse  the 
blade  and  stroke  the  opposite  side 
with  the  same  type  of  motion.  Use 
a  light  even  pressure.  The  wire 
edge  or  burr  that  may  be  left  on  a 
knife  blade  after  whetting  may  be 
removed  by  stropping  both  sides  on 
a  soft  block  of  canvas  or  leather. 

Practice 

Why  is  it  important  to  remove  the 
burr  after  whetting? 


flow  sharpen  your  knife  and  check 
with  your  instructor. 


SHARPENING  A  POCKET  KNIFE 


The  STATEMENT  of  the  steps 
of  the  procedure  is  given 
in  the  first  paragraph , 
but  is  NOT  complete.  The 
angle  between  the  knife 
blade  and  the  sharpening 
stone  is  critical ,  but 
is  not  even  mentioned. 


The  PRACTICE  REMEMBERING 
is  not  present.  The 
"practice"  here  is  the 
wrong  task/content}  and 
has  no  feedback. 


The  picture  at  the  bottom 
of  the  page  might  be  an 
EXAMPLE but  it  does  not 
illustrate  each  step. 


If  the  student  sharpens 
his  knife 3  and  gets 
feedback  from  idle 
instructor 3  this  would 
count  as  PRACTICE  USING. 
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Example  3. 
OBJECTIVE: 


TEST  ITEM: 


Task/Content 

Classification: 

Previous  Remember 
Objective: 


Test  Item: 


Required 

Presentation 

Components: 


Completeness 

Requirements: 


"The  student  will  classify  any  call  sign  according 
to  one  of  the  following  types  —  international 
U.S.  Navy  ship,  international  U.S.  Navy  shore, 
indefinite,  task  organization,  voice,  or  not  a 
valid  Navy  call  sign.  Type  names  will  not  be 
provided  during  classification." 

"For  each  of  the  call  signs  listed  below,  write 
the  type  in  the  space  provided.  If  it  is  not  a 
valid  Navy  call  sign,  write  'invalid'." 

NTQT _ 

T2RA _ 

(etc.) 


Use-unaided  -  Category 

"The  student  will  write  from  memory  the  names  and 
characteristics  of  each  of  the  following  types 
of  call  signs:  international  U.S.  Navy  ship, 
international  U.S.  Navy  shore,  indefinite,  task 
organization,  and  voice." 

"In  the  space  below,  write  the  names  and  defining 
characteristics  of  each  of  the  types  of  call  signs." 


Statement  of  the  defining  characteristics  of  each 
call  sign  type. 


Practice  Remembering  the  characteristics. 

Examples  showing  call  signs  actually  being 
classified  according  to  type. 

Practice  Using  the  characteristics  to  classify 
call  signs  according  to  type. 


Statement:  All  characteristics  must  be  given  for 
each  type  of  call  sign. 

Practice  Remembering:  Recall  of  characteristics 
is  required,  just  as  in  the  Remember  test 
item.  Feedback  must  be  given. 

Examples:  Classification  of  call  signs  according 
to  characteristics  must  be  shown. 

Practice  Using:  Actual  call  signs  must  be  given 
for  the  student  to  classify.  Feedback  must 
be  given. 
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Instruction: 


Note:  This  instruction  continues  over  the  next  11  pages. 


Call  Signs 


One  of  the  most  used  address  designators  is  the  call  sign. 
Every  ship  and  station  has  its  own  "sign"  that  is  used  when 
communicating.  Like  the  other  kinds  of  address  designators, 
call  signs  are  used  for  two  purposes,  calling  and  addressing 
messages. 


(continued  on  following  pages) 
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There  are  six  types  of  call  signs;  each  one  has  its  own 
purpose  and  form.  These  are: 

1.  International  call  signs 

2.  Indefinite  call  signs 

3.  Task  organization  call  signs 

4.  Collective  call  signs 

5.  Net  call  signs 

6.  Voice  call  signs 


International  Call  Signs 

By  international  agreement,  each  nation  is  assigned  a  block 
of  letters  to  assign  to  its  own  ships  and  stations  (civilian 
as  well  as  military).  The  U.S.  is  assigned  the  first  half 
of  the  A's  and  all  the  K's,  W's,  and  N's  for  use  as  call 
signs. 

The  Army  and  Air  Force  begin  their  call  signs  with  "A." 
Commercial  and  private  stations,  along  with  merchant  ships, 
use  the  K's  and  W's.  The  Navy  and  Coast  Guard  use  the  N's 
to  begin  their  call  signs.  That  means  that  the  first  letter 
of  all  international  call  signs  assigned  to  the  Navy  is  the 
letter  "M."  It  is  an  easy  matter  to  determine  whether  a 
Navy  international  call  sign  is  assigned  to  a  ship  or  a 
shore  radio  station— all  shore  radio  stations  use  three-letter 
calls  and  all  ships  use  four-letter  calls. 


PRACTICE 

*  * 

*  *  •k  -k  ★ 

***** 

Circle  each  of  the  international 
that  could  be  assigned  to  the  U. 

call  signs  listed 
S.  Navy. 

1. 

ABDR 

6. 

A1E3 

2. 

NQJI 

7. 

WOWL 

3. 

NABD 

8. 

TTEW 

4. 

NPW 

9. 

NMA 

5. 

QRTW 

10. 

MOS 

*  * 

*  *  *  -k  * 

★  "k  'k  ★  * 
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Uotioe  that  the 
six  call  sign  types 
listed  here  are  not 
the  same  as  in  the 
objective. 


The  STATEMENTS  for 
international  U.S. 

Navy  Ship ,  and  in¬ 
ternational  U.S. 

Navy  Shore ,  are  in 
this  paragraph. 

Ship  =  "N"  -h  3  letters 
Shore=  "N"  +  2  letters 


No  EXAMPLES 

No  PRACTICE  REMEM¬ 
BERING,  but  it  could 
occur  later. 

The  PRACTICE  USING  is 
not  the  same  format 
as  the  final  test 
items . 


FEEDBACK 


This  is  FEEDBACK 
for  the  PRACTICE 
USING  on  the 
previous  page. 


1. 

ABDR 

6. 

AIE3 

2. 

(NQJI) 

7. 

UOWL 

3. 

(nabs) 

8. 

rrEW 

4. 

(Jph) 

9. 

cs> 

5. 

QRTW 

10. 

MOS 

Any  three  or  four  letter  group  that  begins  with  the  letter 
"N"  could  be  an  international  call  sign  for  the  U.S.  Navy. 
Navy  ships  are  assigned  four-letter  call  signs  and  Navy  shore 
radio  stations  are  assigned  three-letter  call  signs.  For 
example,  in  exercise  2,  NQJI  and  NABD  are  international  call 
signs  for  ships— NPW  and  NMA  are  international  call  signs 
for  shore  radio  stations. 


Indefinite  Cali  Signs 

Indefinite  call  signs  are  used  whenever  the  need  arises  to 
conceal  the  identities  of  the  sending  or  receiving  stations. 
To  accomplish  this,  the  codress  form  of  message  is  used. 


This  is  the 
STATEMENT  for 
indefinite 
call  signs. 

"N'>  +  1  letter 
except  "R,  >' 


Instead  of  using  call  signs  in  the  headings  of  codress 
messages,  an  indefinite  call  sign  is  inserted.  This  call 
is  composed  of  two  letters  for  Navy  ships— NA  through  NZ 
(except  for  NR  which  is  not  used  for  this  purpose).  The 
address  portion  of  a  codress  message  is  encrypted  in  the 
text.  This  means  that  the  message  must  be  decoded  in  order 
to  identify  the  sending  and  receiving  commands. 

PRACTICE  ************ 


"NA  t'hro’ugh  NZ" 
might  be  considered 
an  EXAMPLE . 

Circle  the  indefinite  call  signs 
could  be  used  by  naval  commands. 

in  the  list  below  that 

No  PRACTICE 

1. 

NKVD 

6. 

NL 

REMEMBERING }  but 
it  might  occur 
later. 

2. 

3. 

NR 

TN 

7. 

8. 

OL 

NT 

The  PRACTICE 

USING  is  the 

4. 

NB 

y. 

NOP 

wrong  format. 

5. 

NAL 

10. 

P2N3 

***** 

*  * 

***** 
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_  *  -  - 


FEEDBACK 

1.  NKYD 

6.  (h l) 

2.  HR 

7.  01 

3.  TN 

8.(jjT) 

4.(^Nb) 

9.  NOP 

5.  NAL 

10.  P2N3 

Any  two- letter  group  beginning  with  the  letter  "N,"  except 
for  NR,  may  be  used  as  an  indefinite  call  sign. 


Task  Organization  Call  Signs 

Task  organization  call  signs  are  assigned  to  commands  for 
temporary  use  during  tactical  situations.  During  a  tactical 
situation,  ships  communicate  as  one  task  organization.  When 
the  operation  is  completed,  these  call  signs  are  no  longer 
needed  because  the  ships  may  be  reassigned  to  another  force 
which  uses  its  own  call  signs. 

Task  organization  call  signs  are  listed  in  ACP  112(B),  and 
ACP  112  U.S.  SUPP(A)-1 ,  both  titled  "Task  Organization  Call 
Sign  Book."  The  latter  contains  the  call  signs  assigned  to 
the  United  States.  These  books  are  classified  Confidential; 
be  sure  that  you  treat  them  properly. 

Remember  that  international  call  signs  are  composed  of 
letters  only.  However,  task  organization  call  signs  are  made 
up  in  the  following  manner:  letter-number-letter-letter. 

For  example,  task  organization  call  signs  could  be  P70R,  or 
A2CD.  There  are  no  characteristics  for  identifying  task 
organization  call  signs  other  than  this  letter-number-letter- 
letter  arrangement. 


PRACTICE  ************ 

Circle  the  task  organization  call  signs  in  the  list  below 
that  could  be  assigned  to  Naval  commands. 


1. 

P2Q4 

6. 

U4RD 

2. 

KWLH 

7. 

Y87.L 

3. 

NE 

8. 

NOP 

4. 

Q2NY 

9. 

BLUE  SMOKE 

5. 

HOPQ 

*  * 

10. 

***** 

KOU 

*  *  *  * 

This  is  FEEDBACK 
for  the  FRACTICE 
USING  on  the 
prei'ious  page. 


This  is  the  STATEMENT 
for  task  organization 
call  signs. 

Here  are  some 
EXAMPLES , 


No  PRACTICE 
REMEMBERING,  but 
it  might  occur 
later. 


'The  PRACTICE  USING 
is  the  wrong  format. 
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FEEDBACK 

1.  P2Q4 

6.  (^UARd) 

2.  KWLH 

3.  NE 

8.  NOP 

4.^Q2NY) 

9.  BLUE  SMOKE 

5.  NOPQ 

10.  KOU 

The  basic  pattern  for  task  organization  call  siqns  is 
number-letter-letter. 


Note:  This  instruction  continues  over  a  few  more  pages ,  but  ox  this 
point  we  have  some  comments. 

Up  to  now,  there  has  not  been  any  PRACTICE  REMEMBERING. 

At  this  point,  four  types  of  call  signs  have  been  introduced. 
Some  start  with  "N, "  some  have  four  letters,  some  have  different 
patterns  of  letters  and  numbers  —  all  of  them  are  easily 
confused.  Because  of  this ,  Practice  Rememberi-ng  should  have 
occured  by  now.  It  should  be  gust  like  the  remember-level 
test  item ;  that  is,  "write  the  names  and  characteristics  of 
the  types  of  call  signs  that  you  have  read  about. "  Feedback 
would  be  provided. 

Notice  also,  that  while  there  is  practice  and  feedback  at 
the  use  level,  that  practice  is  not  the  same  format  and  does 
not  have  the  same  conditions  as  the  objective  and  test  item, 
and  therefore  is  NOT  CONSISTENT. 


y 

s 


I 

t 


letter- 
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Collective  Call  Signs 


Collective  call  signs  are  used  whenever  a  message  needs  to 
be  sent  to  several  commands  or  ships  instead  of  just  one. 

To  save  the  time  and  effort  that  would  be  needed  to  prepare 
a  message  addressed  to  each  command,  a  collective  call  sign 
is  used.  This  call  sign  is  one  that  automatically  addresses 
a  particular  group  of  commands.  For  example,  the  collective 
call  sign  NABT  means  "All  ships  assigned  Greenland  patrol 
duty." 

Some  collective  call  signs  assigned  to  the  Navy  look  exactly 
like  international  call  signs— in  fact,  they  are  interna¬ 
tional  call  signs  that  have  been  designated  as  collective 
call  signs.  These  are  listed  in  ACP  113,  "CalT  Sign  Book 
For  Ships."  In  section  4  of  ACP  113,  you  will  find  the  coi* 
lective  call  signs  listed,  and  you  will  find  the  same  call 
signs  listed  in  the  section  of  ACP  113  that  lists  interna¬ 
tional  call  signs  for  Navy  ships.  Just  remember  that  a  col¬ 
lective  call  sign  is  an  international  call  sign  that  has 
been  given  a  special  meaning— that  of  addressing  a  specified 
group  of  commands. 

The  Navy  uses  fewer  collective  call  signs  than  other  types 
of  call  signs.  As  they  are  mostly  assigned  to  ships  operat¬ 
ing  in  a  certain  geographic  section  of  the  world,  not  many 
are  needed. 

Just  as  radiomen  must  learn  the  call  sign  for  their  ship, 
they  must  also  learn  the  collective  call  sign  for  the  area 
in  which  their  ship  is  operating.  This  comes  rather  quickly 
since  much  of  the  traffic  sent  to  a  ship  is  addressed  with  a 
collective  call  sign  and  the  radioman  normally  sees  it 
often. 

PRACTICE  ************ 


Collective  call 
signs  are  not  in 
the  objectives  <*nd 
therefore  should  not 
be  in  this  lesson. 

If  you  read  this 
pages  you'll  find 
that  collective 
call  signs  are 
really  international 
call  signs s  and 
you  can't  tell  them 
apai't  except  by 
looking  than  up  in 
ACP  113.  There  is 
a  later  objective  on 
the  use  of  ACP  113 , 
and  this  information 
should  be  postponed 
until  the  instruction 
for  that  objective. 
Here-s  it  just  con¬ 
fuses  the  student. 


Circle  the  collective  call 
could  be  used  by  the  Navy. 

signs 

in  the  list  below  that 

1. 

NPE 

6. 

W2Z8 

2. 

P2ZN 

7. 

NOMV 

3. 

BUCKLE  ON 

8. 

KRLE 

4. 

NPDJ 

9. 

NO 

5. 

NA 

10. 

MTLQ 

***** 

*  * 

***** 

He  don’t  need  this , 
because  it  is  on 
collective  call 
signs. 
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FEEDBACK 


Net  call  signs 
are  not  in  the 
objective  either. 

Again ,  if  you 
read  this ,  you'll 
find  that  net 
call  signs  are 
either  inter¬ 
national  call 
signs ,  or  voice 
call  signs  (which 
we  haven’t  seen 
yet). 

All  this  should 
be  deleted,  and 
there  should  be 
a  later  objective 
on  net  call  signs. 


1.  NPE 

6. 

W2Z8 

2.  P2ZN 

7. 

& 

3.  BUCKLE  ON 

8. 

KRIE 

4.(j}PDj) 

9. 

ND 

5,  NA 

10. 

<^Q) 

International  collective  call  signs  are  call  signs  that  have 
been  designated  for  the  specific  purpose  of  addressing  a 
group  of  commands  in  a  geographic  location.  They  look  ex¬ 
actly  like  international  call  signs  and  you  need  to  use 
ACP  113  in  order  to  tell  a  collective  call  from  an  indivi¬ 
dual  call. 


Net  Call  Signs 

Net  call  signs  are  used  to  address  all  commands/stations 
that  are  using  a  particular  radio  net.  A  net  is  a  group  of 
stations  {not  necessarily  on  different  ships)  that  are  in 
direct  communication  over  one  radio  network.  That  is, 
each  station  can  hear  the  calls  of  all  the  others  and 
each  station  can  call  any,  or  all,  of  the  others.  One 
example  of  a  net  call  sign  is  NQN.  This  means,  "All 
U.S.  Navy  Radio  Stations  guarding  the  ship-shore  high 
frequency  calling  series."  An  example  of  the  use  of  this 
call  sign  would  be  when  a  station  desired  to  communicate 
with  jalji  of  the  stations  at  the  same  time. 

Net  call  signs  are  also  used  on  voice  circuits.  An  example 
of  a  voice  net  call  sign  is  OVERWORK.  This  call  sign  means: 
"All  U.S.  Navy  ships  on  this  circuit."  Again,  a  net  call 
sign  would  be  used  when  the  calling  station  wishes  to  talk 
to  aV[  the  other  stations  at  once. 

These  two  examples  are  similar  to  other  types  of  call  signs. 
This  is  because  they  are  listed  along  with  other  types.  NQN 
is  found  in  the  listing  of  international  call  signs  and 
OVERWORK  is  listed  with  the  voice  call  signs. 

Net  call  signs  should  not  be  confused  with  collective  call 
signs.  Net  calls  always  specify  the  particular  circuit  over 
which  they  are  used.  A  collective  call  designates  certain 
commands  by  name  or  location  (geographic  area).  Net  call 
signs  are  either  international  call  signs  or  voice  call 
signs  so  designated  for  this  use. 
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PRACTICE  ************* 

Check  (/  the  two  items  below  that  describe  a  net  call  sign: 

_  a.  A  call  sign  represents  a  group  of  stations  in 

direct  communication  with  each  other  on  a  common 
channel . 

_  b.  A  call  sign  always  begins  with  the  letter  “N." 

_  c.  A  call  sign  may  be  composed  of  three  letters  or 

pronounceable  words. 

_  d.  A  call  sign  is  used  when  a  message  is  in  codress 

form. 

************ 


j  This  is  p ractice 
;  for  net  call  signs, 

J  so  it  should  be 
'  deleted . 

i 

l 

1  Notice  that  this  is 
j  a  fumy  kind  of 
j  practice  remembering . 
«  It  requires  reccg- 
i  nit ion,  not  recall. 
j  It  should  have  been: 

|  "In  the  space  below, 
write  the  charaa- 
teristics  of  a  net 
call  signi  "  if  it 
was  to  be  included 
at  all. 


FEEDBACK 

The  two  items  you  should  have  checked  which  describe  a  net 
call  sign  are: 

_^a.  A  call  sign  represents  a  group  of  stations  in 

direct  communication  with  e<ich  other  on  a  common 
channel . 

c.  A  call  sign  may  be  composed  of  three  letters  or 
pronounceable  words. 

Remember,  a  net  call  sign  may  be  an  international  or  a  voice 
call  sign  that  has  been  assigned  for  special  use.  Net  call 
signs  look  like  international  or  voice  call  signs,  and  their 
use  can  be  determined  only  by  consulting  the  proper  publica¬ 
tions. 


We  put  vhis  here 
to  save  space.  Tt 
really  appears  on 
a  separate  page. 
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Voice  Call  Signs 


Voice  call  signs,  as  the  name  implies,  are  used  over  voice 
radio  circuits.  These  consist  basically  of  words  or  com¬ 
binations  of  words  that  can  be  easily  pronounced.  For 
example,  the  Navy  ship  U'SS  Jason  is  assigned  the  voice  call 
sign  INKSTAND.  This  is  a  word  that  is  easily  pronounced, 
and  like  the  call  signs  discussed  earlier,  it  is  only 
assigned  to  one  command/ship. 

Just  as  there  are  collective  international  call  signs,  there 
are  also  collective  voice  call  signs.  There  is  no,  differ¬ 
ence  in  the  way  collective  voice  call  signs  and  those  for  a 
single  command  look.  However,  you  need  the  same  publica¬ 
tion,  JANAP  119(F),  "Joint  Voice  Call  Sign  Book"  in  order  to 
decode  and  encode  both  types. 

JANAP  119(F)  is  classified  Confidential  because  the  dis¬ 
closure  of  some  call  signs  and  the  commands  they  address  is 
a  violation  of  security. 


All  voice  call  signs  fall  into  four  types,  each  of  which  is 
listed  in  JANAP  119(F).  These  are: 


This  is  the 
STATEMENT  for 
voice  call  signs. 


1.  Word  (or  words) 

2.  Word  (or  words)  plus  a  letter 

3.  Word  (or  words)  plus  a  digit 

4.  Digit  and  word  (or  words) 


Here  are  some 
EXAMPLES . 


No  PRACTICE 
REMEMBERING. 


This  PRACTICE 
USING  is  the 
wrong  format. 


Some  examples  are:  INKSTAND,  END  RUN  A  (ALPHA),  HAMLET  8 
(EIGHT),  and  6  (SIX)  WATERSHED.  (Note  that  letters  and 
digits  are  pronounced  as  words).  Of  these  four  types  the 
last  one  (digit  and  word  or  words)  is  a  special  type  that 
is  reserved  for  special  use;  you  may  seldom  have  a  chance  to 
use  it. 


PRACTICE  *  *  * 

*  * 

*  * 

*  *  *  *  ■ 

Circle 

the  voice  call 

signs 

in  the  list  below. 

1. 

BLUE  BOTTLE 

6. 

DSWN 

2. 

RUWSADF 

7. 

SEA  BOY  A 

3. 

USS  RONKARIS 

8. 

AIG  402 

4. 

NCCD 

9. 

SEX  APPEAL  1 

5. 

P7QR 

10. 

NA 

*  *  * 

*  * 

*  * 

*  *  +  * 

/ 
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FEEDBACK 


l.  (^uITbottle) 

6. 

DSWN 

2.  RUWSADF 

7.< 

^SEA  BOY  A  (ALPHAT) 

3.  USS  RONKARIS 

8. 

AIG  402 

4.  HCCD 

9.1 

fSEX  APPEAL  1  (GNeT^ 

5.  P7QR 

10. 

NA 

Voice  call  signs  are  always  made  up  of  words,  digits,  or 
letters  that  are  pronounceable  (letters  are  always  pro¬ 
nounced  phonetically,  alpha,  bravo,  Charlie,  et**.,). 


FINAL  PRACTICE 


****** 

*  * 

*  * 

*  * 

You  should  now  be  able  to  identify  by  their  appearance, 
the  five  types  of  call  signs  listed  below.  Match  each  type 
of  call  sign  in  column  A  with  the  call  signs  in  column  B. 
Note  that  there  is  more  than  one  type  of  some  call  signs  and 
*  that  some  items  in  column  B  are  not  valid  call  signs. 

A 

B 

1  1. 

International 

_  A. 

ROB INH00D  3 

(U.  S.  Navy  ships) 

_  B. 

NG 

’  2. 

International  (U.  S.  Navy 
shore  radio  stations) 

_  c- 

NPO 

_  D. 

DESRON  FOUR 

% 

_  E. 

S3BR 

3. 

Task  Organization 

_  F» 

NEAC 

4. 

Indefinite 

_  G- 

N7GL 

5. 

Voice 

_  H. 

NBRA 

6. 

This  is  not  a 

_  I. 

NPQL 

1 

valid  call  sign. 

_  J. 

NL 

__  K. 

WAY  OUT  PAPA 

: 

_  L. 

COMNAVAIRPAC  TWO 

, 

_  M. 

NGA 

i 

t 

_  N. 

NAPD 

_  o. 

OVERLOAD 

****** 

*  * 

*  * 

*  * 
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This  final  practice 
is  supposed  to  be 
PRACTICE  USING.  It 
is  similar  to  the 
use-level  test  items , 
except  that  it  is 
also  the  wrong  format. 

This  item  is  a 
matching  item,  and 
gives  away ,  in 
column  A ,  the 
category  names. 

It  should  have  been: 
"For  each  of  the 
call  signs  listed 
below ,  write  the 
type  in  the  space 
provided.  If  it 
is  not  a  valid 
Navy  call  sign, 
write  *  invalid ' . " 


FEEDBACK 


5 

_  A. 

ROB INHOOD  3 

1 

_  1. 

NPQL 

This  is  the 

4 

_  B. 

NG 

4 

_  J- 

NL 

FEEDBACK  for 
the  filial 
practice. 

2 

_  C* 

NPO 

5 

..  K* 

WAY  OUT  PAPA 

6 

..  D- 

DESRON  FOUR 

6 

_  L. 

COMNAVAIRPAC  TWO 

3 

_  E. 

S3BR 

2 

_  M. 

NGA 

1 

_  F. 

NEAC 

1 

„  N. 

NAPD 

3 

_  G- 

N7GL 

5 

_  0. 

OVERLOAD 

1 

H. 

NBRA 

Two  of  these  are  not  valid  call  signs;  items  D  and  l  are 
plain  language  addresses.  Before  you  continue  with  the 
lesson  be  sure  that  you  understand  the  pattern  for  each  type 
of  call  sign: 


This  is  a  pretty 
good  summary 
of  the 
STATEMENTS 
for  each 
type  of 
call  sign. 


*  Voice  call s— pronounceable  word(s),  digit(s),  and 
letter(s) 

*  International  call  signs 

(ships)— four  letters  beginning  with  "N" 

(shore  radio  stations)— three  letters  beginning  with 
"N" 


(indefinite) — two  letters  beginning  with  "N" 

(except  NR) 

*  Task  organization  call  signs — letter-nurnber-letter- 
letter  pattern 


CONSISTENCY  SUMMARY 


STATEMENTS:  Statements  were  present  and  complete  for  each  of  the 

five  types  of  call  signs  given  in  the  objective,, 

PRACTICE  REMEMBERING:  No  Practice  Remembering  anywhere. 

EXAMPLES:  Examples  were  given  for  two  of  the  types — task 

organization  and  voice.  The  examples  were  not  really 
complete 3  because  they  did  not  illustrate  how  the 
defining  characteristics  were  used  to  classify. 


PRACTICE  USING:  Pi'actice  Using  cannot  be  counted  as  present  or  complete , 

because  the  wrong  format  was  used  throughout.  The 
practice  did  have  feedback 3  however. 


a- 

O 


U 
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Chapter  5 

PRESENTATION  ADEQUACY 


Introduction 

In  the  last  chapter,  the  consistency  of  the  instructional 
presentation  with  objectives  and  test  items  was  assessed.  That  is, 
the  presentation  was  checked  to  make  sure  that  the  right  combination 
of  instructional  components  was  present.  This  chapter  is  concerned 
with  the  adequacy  of  the  instructional  components. 

Instruction  can  be  consistent,  but  still  not  teach  as  effectively 
as  it  could.  The  adequacy  procedures  incorporate  a  number  of  instructional 
design  principles  which  have  been  shown  to  promote  student  learning. 

These  include  guidelines  for  formatting  information  so  students  can 
find  it,  for  communicating  it  clearly  and  effectively  so  students  can 
understand  it,  and  for  including  additional  explanation, 
so  students  can  better  learn  and  remember  the  information. 


On  the  next  two  pages,  the  PRESENTATION  ADEQUACY  procedures 
from  Volume  IV  are  presented.  Then  additional  explanation  and 
examples  are  given. 


STEP  1: 


PRESENTATION  ADEQUACY 


Instructional 

Quality 

Inventory 


DETERMINE  whether  or  not  each  required  PRESENTATION  COMPONENT  meets  the  general 
TTCE-'VACY  criteria  below. 

SEPARATED:  Statements,  Examples,  or  Practice  must  be  SEPARATED  from  the 

rest  of  the  Instruction.  There  are  different  Days  components 
nay  be  iterated: 


a.  Tet  off  the  component  with  a  box. 

b.  Vse  a  different  color  or  type  face,  or  underline, 

c.  Place  on  a  separate  page,  or  in  a  special  place  on  the  page. 

d.  For  audio,  nodes,  or  lectures,  pause  before  introducing  the 
component. 


IDENTIFIED: 


Statements,  Examples,  and  Practice  must  be  IDENTIFIED  so  the 
student  knows  what  they  are,  and  can  locate  them.  Labels  can 
be  used  to  identify  different  components: 


Definition  of  ...  Example  Practice 

Procedure  for  . . .  Demonstration  Test  Yourself 

Key  Point: 


CLEARLY  STATED:  Statements,  Examples,  and  Practice  must  be  CLEARLY  STATED  so  the 
student  can  understand  them.  The  following  criteria  should  be 
used: 


a.  The  READING  LEVEL  must  be  appropriate  for  the  students. 

b.  The  presentation  should  not  be  confusing,  vague,  or  too  wordy. 

c.  All  essential  information  should  be  present ;  the  student 
should  not  be  referred  to  other  places  to  obtain  information. 

d.  All  presentations  should  be  PERFORMANCE-ORIENTED ,  not 
topic-oriented. 


STEP  2:  DETERMINE  whether  or  not  each  required  PRESENTATION  COMPONENT  meets  the  specific  ADEQUACY 
criteria  below: 


STATEMENTS:  HELP:  In  addition  to  the  statement,  the  instruction  should  include  something 

to  help  the  student  better  understand  and  remember  the  statement. 

Methods  of  prodding  help  include: 

a.  Cidng  a  MNEMONIC  (memory  trick)  or  other  memory  aid. 

b.  Representing  the  statement  with  pictures,  symbols,  flowcharts,  etc. 

c.  Explaining  how  the  statement  relates  to  something  the  student 
already  knows,  how  it  fits  in  the  course,  why  it  is  important. 

d.  Gidng  more  explanation  about  what  the  statement  means. 

CLEARLY  STATED:  In  addition  to  the  criteria  above,  statements  for  different  content  types 
must  meet  the  criteria  below: 


(Continued  on  other  side) 


CATEGORIES:  Give  a  decision  rule  or  search  strategy  for  classification. 
PROCEDURES:  Each  step  should  have  only  one  action. 

RULES:  If  a  formula  is  used,  symbols  must  be  defined. 


EXAMPLES: 


HELP: 


Fxamples  should  include  additional  infonnation  to  help  the  student 
understand.  Sene  types  of  HElp  for  different  COETEHT  PIPES  are  given  telw 

HELP  for  CATEGORIES:  Highlight  critical  characteristics. 

Explain  the  reasons  for  classification. 

Illustrate  the  uce  of  the  decision  rule 
or  tear  oh  strategy. 

HELP  for  PROCEDURES  or  RULES:  Explain  the  steps. 

Give  more  infomation  on  het!  to  per  fern 
each  step. 

Give  a  demonstration  of  each  atep. 

Give  flciuharta,  tables,  etc. 

HELP  for  PRIRCIFLES:  Highlight  important  causes,  effects,  relationships. 

Give,  additional  information  about  hov  the  principle 
applies,  or  vhy  it  doesn’t. 

MATCHING:  Examples  should  be  matched  with  non-examples  according  to  the  presence 

(categories  only)  an<*  absence  of  each  critical  characteristic. 

EASY  to  ‘SARD:  Early  examples  should  be  simplified.  Conditions  and  standards  may  be 

relaxed  on  early  examples  so  the  student  can  learn  gradually. 

ENOUGH  EXAMPLES:  There  should  !  a  enough  examples  to  cover  the  content  area  thoroughly. 

For  CATEGORIES:  At  least  one  example-nonexatrple  uair  for  each 
critica1  characteristic. 

For  PROCEDURES  or  RULES:  One  example  for  each  important  step. 

example**  for  all  possible  branches  or  decisions. 

COMMON  ERRORS:  Examples  should  show  why  common  errors  are  wrong. 


PRACTICE 

REMEMBERING 

or 

PRACTICE 

USING: 


FREE  OF  HINTS:  The  practice  items  should  not  include  anything  that  won't  appear  on 

the  test,  or  on  the  job. 

EASY  to  HARO:  (USING  only)  Early  practice  items  may  be  simplified,  or  conditions  and 

standards  may  be  relaxed.  Final  practice  items  should  be  typical  of  the 
job. 

COMMON  ERRORS:  (USING  only)  Practice  should  include  the  opportunity  for  common  errors, 

so  they  can  be  corrected. 

ENOUGH  PRACTICE:  There  should  be  enough  practice  so  the  student  has  an  adequate  opportunity 
to  learn. 

For  CATEGORIES:  Practice  for  presence  or  absence  of  all 
critical  characteristics. 

For  PROCEDURES  or  RULES:  Practice  for  all  possible  branches  or 
decisions. 

FEEDBACK:  The  FEEDBACK  must  be  SEPARATED  and  IDENTIFIED. 

FEEDBACK  HELP:  The  FEEDBACK  should  include  HELP  like  tne  help  for  examples. 
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Additional  Explanation  for  the  PRESENTATION  ADEQUACY  Procedure. 


During  the  Presentation  Consistency  procedure,  the  locations  of 
each  of  the  required  presentation  components  for  each  objective  should 
have  been  noted.  The  ADEQUACY  procedure  should  be  applied  to  each  of 
these  presentation  components. 

STEP  1:  The  criteria  in  step  1  should  be  applied  to  all  presentation 
components. 

The  "separated"  and  "identified"  cri.teria  are  formatting 
requirements.  f<'c  student  needs  to  be  able  to  find  each 
presentation  con  'f'ent,  and  to  know  what  he  is  looking  at. 

The  "clearly  sxa  ■ed"  criteria  are  among  the  most  important 
in  the  IQI: 

a.  There  are  several  "reading  level"  formulas  that  can 

be  used  to  judge  reading  ease,  but  they  don't  indicate 
how  well  material  can  be  understood.  The  best  way  to 
see  if  reading  material  is  appropriate  is  to  consider 
the  material  from  the  student's  point  of  view. 

b.  At  the  same  time,  consider  whether  or  not  the  components 
are  confusing,  vague ,  or  too  wordy. 

c.  Essential  information ,  like  charts,  tables,  or  ex¬ 
planations,  should  be  presented  in  conjunction  with 
the  related  component.  The  student  should  not  have 
to  interrupt  his  study  to  go  find  information. 

d.  All  presentations  should  be  performance  oriented  in  two 
ways.  First,  a  presentation  should  be  directly  related 
to  what  the  student  will  do  after  the  courses  again, 
REMEMBER  THE  JOB.  All  too  often .  presentations  take 
the  form  of  "everything  you  wanted  to  know  about  . . .," 
with  no  information  about  what  the  job  is  really  like, 
or  why  the  information  is  important.  Instead,  presen¬ 
tations  should  be  written  in  terms  of  the  job  to  be 
done.  Orienting  instruction  to  job  performance  is  done 
differently  for  different  presentation  components. 
Statements  give  essential  information,  and  are  not 

job  oriented  by  themselves.  The  job  orientation,  there¬ 
fore,  is  given  by  providing  additional  information  about 
the  job  context  in  which  the  statement  will  be  applied. 
(See  "Statement  Help"  under  step  2  below.)  Examples 
and  practice  items  are  job  oriented  if  they  are  taken 
from  actual  job  situations. 

Second,  a  presentation  should  also  orient  the 
student  to  the  performance  expected  during  the  instruc¬ 
tion.  This  means  that  the  student  must  be  told,  what 
is  in  the  instruction,  how  best  to  study  it,  and  what 
the  tests  will  be  like.  This  is  often  accomplished 
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by  giving  the  objective (s)  at  the  beginning  of  the 
presentation.  Unfortunately,  objectives  are  not 
usually  written  to  ccmnaiicate  directly  to  the 
student.  They  can  be  confusing,  because  they  con¬ 
tain  unfamiliar  words  and  topics,  and  complicated 
sentences.  Instead,  the  instructional  orientation 
should  tell  the  student  what  is  in  the  presentation, 
what  he  should  do  with  the  information  !e.  g.t  memorize 
it),  and  how  he  will  be  tested. 


STEP  2:  The  criteria  in  step  2  apply  to  each  different  presentation 

component  individually. 

STATEMENTS:  Help:  Several  types  of  "help"  for  statements 

are  described  in  seep  2. 

Mnemonics  are  most  often  used  wi th 
Remember-Pact  information,  but  they 
should  also  be  used  for  other  content 
types  at  the  Remember  level.  Methods 
of  constructing  Tsaamonics  are  described 
in  TAEG  Report  #60,  Use  of  Mnemonics  in 
Training  Materials :  A  Guide  for  Technical- 
Writers,  by  Braby  and  Kincaid  (1078). 

Since  different  people  learn  in  different 
ways ,  it  is  often  desirable  to  include 
more  than  one  representation  of  a  statement. 
Pictures ,  symbols,  flowcharts,  tables,  esc., 
can  be  used. 

Again,  a  job-relevant  context  must  be 
provided  for  a  statement.  This  context 
should  include  a  description  of  the 
job  environment,  how  the  information  will 
be  applied  on  the  job,  and  reasons  why  it 
is  important  to  learn  the  information. 

This  additional  context  helps  motivate 
the  student. 

Clearly  Stated:  For  difficult  Category  tasks,  it  is  often 
desirable  to  include  :u?t  only  the  Category 
characteristics,  but  also  a  decision  strategy 
or  rule  for  making  classifications .  For 
example,  in  chapter  2  on  task/content 
classification,  we  gate  characteristics  of 
task  levels  and  content  types,  and  we  also 
gave  a  decision  strategy:  "First  decide 
if  its  Remember  or  Use j  then  if  its  Use, 
decide  if  its  aided  or  unaided ;  then  decide 
on  the  content  type. " 

For  Procedures,  each  step  should  have  only 
one  action.  If  there  is  more  than  one,  the 
step  should  be  broken  up.  Also,  explicit 
safety  precautions  should  be  included  as 
separate  steps. 
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EXAMPLES: 


Help:  Again ,  examples  should  be  taken  from 

situations  typical  of  the  gob  to  be 
performed  after  training ,  so  that 
the  presentation  is  performance 
oriented.  Help  for  the  exarp'tes  can 
also  maintain  the  perfoimance 
orientation. 

Matching:  Matching  can  be  used  with  the  CATEGORY 

content  type  to  illustrate  why  one 
object ,  event ,  or  idea  is  a  member 
of  the  category  and  a  simihzt'  one  is  not. 
.'Etching  is  usually  helpful  when  the 
category  has  a  large  number  of  critical 
characteristics,  or  when  things  that  are 
members  of  the  category  are  easily 
confused  with  things  that  are  not. 

Matching  is  done  by  changing  only  one 
critical  characteristic  at  a  time.  This 
method  demonstrates  why  something  is 
a  member  of  the  category  when  the 
characteristic  is  present  and  why  it 
is  not  a  member  when  it  is  changed. 

Easy  to  Hard:  In  order  to  teach  someone  how  to  perform 
a  complex  task,  it  is  often  necessary 
to  make  it  easier  to  do  at  the  beginning 
of  learning.  To  do  this,  the  conditions 
or  standards  can  be  relaxed.  The  action, 
however ,  should  never  be  changed. 

For  example,  the  task  "field-strip  an 
M-16  rifle  in  total  darkness"  is  taught 
initially  in  the  light. 

Enough  Examples:  Some  guidelines  for  examples  are  given 

in  step  2.  The  decision,  however,  is 
usually  a  matter  of  judgment,  and  the 
advice  of  a  subject-matter  expert  should 
be  obtained. 

\fhen  generalization  is  required  (categories, 
rules,  principles) ,  more  examples  need  to 
be  given.  Furthermore,  the  examples  should 
have  a  range  of  difficulty  typical  of  the 
job. 

Common  Errors:  If  there  are  errors  that  are  typically 

made  on  the  job ,  examples  on  these  should 
be  included.  An  explanation  of  how  the 
errors  can  be  avoided  should  also  be 
included. 
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PRACTICE 
REmSERING 
or 

PRACTICE 
USING 

Easy  to  Hard:  The  same  criteria  given  for 

examples  apply  to  practice  items. 
Again,  the  conditions  or  standards 
may  be  relaxed  on  early  practice 
items ,  blit  the  action  should 
not  be  chayiged. 

Common  Ei'rors :  Practice  items  should  include 
the  opportunity  for  students  to 
make  common  errors.  The  feedback 
should  then  clearly  show  why  the 
error  is  wrong,  and  how  to  avoid 
it. 

Enough  Practice:  The  same  criteria  given  for 
examples  apply  to  practice. 

Feedback:  /5s  a  minimum,  students  must  be 
told  whether  they  were  right  or 
wrong  on  each  practice  item ,  and, 
if  wrong 3  what  the  correct  answer 
is.  For  performance  tasks  in  which 
a  step-by-step  process  is  required , 
feedback  should  he  given  for  each 
step. 

The  purpose  of  feedback  is  to  help 
the  student  learn  the  material. 
Therefore 3  the  feedback  should  be 
designed  so  that  the  student  is  led 
lo  resludy  i'rie  information  to  deter¬ 
mine  the  correctness  of  his  answer. 

Feedback  Help:  Students  sometimes  make  errors  because 
they  did  not  learn  from  the  original 
presentation.  Therefore 3  when  pos¬ 
sible ,  feedback  should  include  ad¬ 
ditional  information  or  a  different 
version  of  the  information  than  was 
originally  presented. 

Finally 3  once  feedback  is  given  for 
a  practice  using  item 3  it  becomes 
an  example.  Therefore 3  all  of  the 
criteria  given  above  for  examples 
also  apply  here. 


Free  of  Hints:  The  practice  ttems  should  be¬ 
like  the  test.  Hints  should 
be  included  in  the  feedback  for 
the  practice  items ,  riot  in  the 
items  themselves. 


General  Garments  about  Consistency  and  Adequacy: 


The  criteria  given  in  this  chapter ,  and  the  previous  one,  apply 
as  well  to  nonpi”".»ed  instruction.  Lectures ,  slide- tape  presentations , 
movies ,  videotapes,  and  computer-assisted  instruction  lessons  must  also 
be  consistent  with  the  objectives  and  test  items,  arid  must  be  adequate. 

For  example,  a  good  lecture  will  contain  the  appropriate  combination 
of  presentation  components,  and  they  will  all  be  complete.  The  lecturer 
will  separate  and  identify  the  components,  match  his  presentation  level 
to  his  audience,  make  his  presentation  performance  oriented,  give  emugh 
examples  and  practice,  and  provide  appropriate  feedback.  Therefore,  the 
IQI  can  be  used  to  rate  iristructors  on  how  consistent  their  instruction 
is,  and  how  adequately  they  deliver  it. 


EXAMPLES 


The  example  instructional  presentations  from  the  last  chapter 
are  repeated  in  this  section.  This  time,  they  are  rated  for  adequacy. 
For  each  example,  we  will  determine  whether  or  not  (1)  the  presentation 
components  are  separated,  identified,  and  clearly  stated,  (2)  help  is 
given,  and  (3)  examples  and  practice  arc  adequate.  Again,  our  comments 
about  adequacy  will  be  given  in  italics  in  the  margins. 

After  the  examples,  we  have  included  a  revised  version  of  each 
of  the  instructional  presentations.  The  revised  versions  will  be 
consistent  and  adequate. 

The  objectives  and  test  items  for  these  examples  are  given  in 
the  previous  chapter,  and  are  not  repeated  here. 


EXAMPLE  1. 


Instruction: 


The  STATEMENT  in 
the  picture  at  the 
bottom  of  the  puge 
is  separated ,  but 
not  identified. 

The  reading  level 
is  probably  too 
difficult. 

The  presentation 
is  topic  oriented, 
not  oriented  to 
the  job  the  student 
will  do. 

The  presentation 
does  not  give  the 
student  any  "how¬ 
to-study"  infor¬ 
mation.  How  is 
the  student  to 
know  what  is  im¬ 
portant  to  remember 
in  this  presentation? 
The  student  should 
be  told  what  he  will 
have  to  learn  or 
memorize  for  the 
test. 

There  is  no  real 
help  for  the  statement. 
A  mnemonic  could 
have  been  given  to 
make  the  part  names 
easier  to  remember. 

There  is  no  Practice 
Remembering,  so  the 
student  has  no  way 
to  determine  what 
to  study,  or  how 
well  he  is  learning. 


Pumps 

Pumps  provide  the  force  required  to 
move  fluids.  They  do  this  by  establishing 
and  maintaining  a  difference  in  pressure 
between  the  inlet  and  (discharge)  outlet 
of  the  pump.  Did  you  know  that  you  carry 
a  pump  around  with  you?  Ail  the  time? 

Just  like  your  own  heart,  the  "heart"  of 
most  fluid  transfer  systems  is  a  pump. 

Aboard  ship,  pumps  are  used  to  move 
water,  oil,  fuel,  and  air.  These  fluids 
and  gases  are  moved  by  pumps  into  and  out 
of  storage  tanks,  piping  systems,  and 
machinery  to  move  the  ship,  make  elec¬ 
tricity,  and  keep  the  crew  safe  and 
comfortable.  Without  its  pumps,  a  steam 
or  diesel-powered  ship  couldn’ t  even  get 
underway. 

There  are  many  different  kinds  of 
pumps.  A  basic  reciprocating  pump  is  a 
pump  which  is  used  to  push  air  into  various 
receptacles  such  as  a  basketball  or  a 
bicycle  tire.  This  is  accomplished  by 
having  the  connecting  rod  (which  connects 
the  piston  to  an  outside  energy  source) 
push  the  piston  (plunger- shaped  mechanism) 
down,  forcing  air  out  of  the  cylinder 
(which  acts  as  the  encasement  for  the 
piston  and  air)  and  through  a  valve. 

The  purpose  of  the  valve  in  a  basic 
reciprocating  pump  is  to  enable  gas 
(e.g.,  air)  to  enter  and  escape  from 
the  cylinder. 

Presented  below  is  a  diagram  which 
illustrates  the  major  parts  of  basic 
reciprocating  pumps. 
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EXAMPLE  2. 


Instruction: 


The  STATEMENT  is  separated 
and  identified:,  but  is  not 
clearly  stated.  It  would 
be  better  to  list  each 
step  on  a  separate  line , 
and  each  step  should  have 
only  one  action.  There 
is  no  help  for  the  state¬ 
ment,  and  there  are  no 
instructions  to  the 
student  about  what  to 
learn. 

There  is  no  Practice 
Remembering,  so  adequacy 
cannot  be  evaluated. 

The  EXAMPLE  is  separ  .  d 
and  identified,  but  is 
not  adequate  because  it 
does  not  illustrate  each 
important  step  of  the 
procedure.  There  is  no 
help  in  the  example. 

A  better  example  would 
have  been  either  a 
demonstration  of  each 
step  by  the  instructor, 
or  at  least  a  separate 
picture  for  each  important 
step. 

The  PRACTICE  USING  is  not 
identified  as  practice. 

Also,  its  adequacy  really 
depends  on  how  the  instructor 
supervises  and  provides 
feedback. 

For  performnce  orientation 
for  this  instruction,  we 
would  expect  to  see  some 
information  about  why  sharp 
knives  are  important,  and 
how  they  are  used  on  the 
yob.  Also,  the  student 
should  be  told  what  he  will 
have  to  do  to  score  well  on 
the  test. 


Sharpening  a  pocket  knife 
on  a  sharpening  stone: 

Most  pocket  knives  may  be 
sharpened  on  a  medium  or  fine 
grade  sharpening  stone  with  a 
few  drops  of  oil  spread  on  the 
surface.  Hold  the  handle  of  the 
knife  in  one  hand,  and  place  the 
blade  across  the  stone.  Press  the 
blade  down  with  the  fingers  of  the 
other  hand,  and  stroke  the  blade 
in  a  circular  motion  as  shown  below. 
After  several  strokes,  reverse  the 
blade  and  stroke  the  opposite  side 
with  the  same  type  of  motion.  Use 
a  light  even  pressure.  The  wire 
edge  or  burr  that  may  be  left  on  a 
knife  blade  after  whetting  may  be 
removed  by  stropping  both  sides  on 
a  soft  block  of  canvas  or  leather. 

Practice 

Why  is  it  important  to  remove  the 
burr  after  whetting? 


Now  sharpen  your  knife  and  check 
with  your  instructor. 


SHARPENING  A  POCKET  KNIFE 
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EXAMPLE  3. 

Instruction: 

Note:  This  instruction  continues  over  the  next  11  paces 


Call  Signs 


One  of  the  most  used  address  designators  is  the  call  sign. 
Every  ship  and  station  has  its  own  "sign"  that  is  used  when 
communicating.  Like  the  other  kinds  of  address  designators, 
call  signs  are  used  for  two  purposes,  calling  and  addressing 
messages. 


(continued  on  following  pages) 


There  are  six  types  of  call  signs;  each  one  has  its  own 
purpose  and  form.  These  are: 

1.  International  call  signs 

2.  Indefinite  call  signs 

3.  Task  organization  call  signs 

4.  Collective  call  signs 

5.  Net  call  signs 

6.  Voice  call  signs 


These  six  types 
are  not  the  same 
as  in  the  objective. 
This  misleads  the 
student.  Orientation 
information  should 
tell  the  student  what 
to  8 tudy,  and  how  he 
will  be  tested. 


International  Call  Signs 


By  international  agreement,  each  nation  is  assigned  a  block 
of  letters  to  assign  to  its  own  ships  and  stations  (civilian 
as  well  as  military).  The  U.S.  is  assigned  the  first  half 
of  the  A's  and  all  the  K's,  W's,  and  N's  for  use  as  call 
signs. 

The  Army  and  Air  Force  begin  their  call  signs  with  "A*11 
Commercial  and  private  stations,  along  with  merchant  ships, 
use  the  K's  and  W's.  The  Navy  and  Coast  Guard  use  the  n's 
to  begin  their  call  signs.  That  means  that  the  first  letter 
cf  all  international  call  signs  assigned  to  the  Navy  is  the 
letter  "H."  It  is  an  easy  matter  to  determine  whether  a 
Navy  international  call  sign  is  assigned  to  a  ship  or  a 
shore  radio  station--all  shore  radio  stations  use  three-letter 
calls  and  all  ships  use  four-letter  calls. 

PRACTICE  ************ 

Circle  each  of  the  international  call  signs  listed  below 
that  could  be  assigned  to  the  U.  S.  Navy. 


This  is  all  topic 
oriented,  rather  than 
performance  oriented. 
The  student  should  be 
told  what  he  needs  to 
learn,  and  how  it  will 
be  used  on  the  job. 


The  STATEMENTS  for 
international  U.S. 
Navy  Ship  and  Shore 
call  signs  are  buried 
in  this  paragraph. 
They  are  not  even  in 
clear  sentences,  let 
alone  separate  and 
identified. 


i. 

ABDR 

6. 

A1E3 

2. 

NQJI 

7. 

WOWL 

3. 

NABD 

8. 

TTF.W 

4. 

NPW 

9. 

NMA 

5. 

QRTW 

10. 

KiOS 

*  * 

**,*** 

*  * 

EXAMPLES  should  have 
been  used  to  show  the 
category  characteristics. 
For  example: 


"NABD"  starts  with  :'N " 
and  has  Z  more  letters 
after  the  N,  so  it  is 
a  U.  S.  Navy  Ship. 


There  is  no  PRACTICE  RE¬ 
MEMBERING  and  the  PRACTICE 
USING,  although  separate  and 
identified,  is  not  consis¬ 
tent  and,  therefore,  can¬ 
not  be  evaluated  for 
adequacy 


The  FEEDBACK  is 
separate  and 
identified  and 
includes  HELPS. 


} 


The  STATEMENT  for 
indefinite  call 
signs  is  in  the 
middle  of  the 
second  paragraph. 

It  is  clearly 
not  separate  and 
identified.  Again , 
the  sentences , 
are  not  clear 
and  the  reading 
level  is  probably 
tod  high. 

Again ,  the 
EXAMPLES  don't 
show  category 
charaoteristics . 

There  is  no 
PRACTICE  REMEMBERING 
and,  again,  the 
PRACTICE  USING 
is  not  consistent. 


FEEDBACK 

1.  ABDR 

6. 

A1E3 

2.  (jiqjj) 

7. 

HOWL 

3.  (jIABQ) 

8. 

TTEW 

4.  (nPvT) 

9. 

C®) 

5.  QRTW 

10. 

MOS 

Any  three  or  four  letter  group  that  begins  with  the  letter 
"N"  could  be  an  international  call  sign  for  the  U.S.  Navy. 
Navy  ships  are  assigned  four-letter  call  signs  and  Navy  shore 
radio  stations  are  assigned  three-letter  call  sinns.  For 
example,  in  exercise  2,  NQOI  and  NABD  are  international  call 
signs  for  ships— NPW  and  NMA  are  international  call  signs 
for  shore  radio  stations. 


Indefinite  call  signs  are  used  whenever  the  need  arises  to 
conceal  the  identities  of  the  sending  or  receiving  stations. 
To  accomplish  this,  the  codress  form  of  message  is  used. 


Instead  of  using  call  signs  in  the  headings  of  codress 
messages,  an  indefinite  call  sign  is  inserted.  This  call 
is  composed  of  two  letters  for  Navy  ships— NA  through  NZ 
(except  for  NR  which  is  not  used  for  this  purpose).  The 
address  portion  of  a  codress  message  is  encrypted  in  the 
text.  This  means  that  the  message  must  be  decoded  in  order 
to  identify  the  sending  and  receiving  commands. 

PRACTICE  ************ 

Circle  the  indefinite  call  signs  in  the.  list  below  that 
could  be  used  by  naval  commands. 


1. 

NK.VB 

6. 

NL 

2. 

NR 

7. 

0L 

3. 

TN 

8. 

NT 

4. 

NB 

9. 

NOP 

5. 

NAL 

10. 

P2N3 

***** 

*  * 

*  * 
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FEEDBACK 

1.  NKVD 

6.® 

2.  NP. 

7.  OL 

3.  TN 

8.© 

4.  (© 

9.  NOP 

5.  NAL 

10.  P2N3 

Any  two-letter  group  beginning  with  the  letter  "N,"  except 
for  NR,  may  be  used  as  an  indefinite  call  sign. 


Task  Organization  Call  Signs 

Task  organization  call  signs  are  assigned  to  commands  for 
temporary  use  during  tactical  situations.  During  a  tactical 
situation,  ships  communicate  as  one  task  organization.  When 
the  operation  is  completed,  these  call  signs  are  no  longer 
needed  because  the  ships  may  be  reassigned  to  another  force 
which  uses  its  own  call  signs. 

Task  organization  call  signs  are  listed  in  ACP  112(B),  and 
ACP  112  U.S.  SUPP(A)-1,  both  titled  '‘Task  Organization  Call 
Sign  Book."  The  latter  contains  the  call  signs  assigned  to 
the  United  States.  These  books  are  classified  Confidential; 
be  sure  that  you  treat  them  properly. 

Remember  that  international  call  signs  are  composed  of 
letters  only.  However,  task  organization  call  signs  are  made 
up  in  the  following  manner:  letter-number-letter-letter. 

For  example,  task  organization  call  signs  could  be  P70R,  or 
A2CD.  There  are  no  characteristics  for  identifying  task 
organization  call  signs  other  than  this  letter-number-letter- 
letter  arrangement. 


PRACTICE  *  *  *  *  *  k  *  *  *  *  x  * 

Circle  the  task  organization  call  signs  in  the  list  below 
that  could  be  assigned  to  Naval  commands. 


1. 

P2Q4 

6. 

U4RD 

2’. 

KWLH 

7. 

Y8ZL 

3. 

NE 

8. 

NOP 

4. 

Q2NY 

0 
•s  • 

BLUE  SMOKE 

5. 

N0PQ 

10. 

KOU 

*  j< 

***** 

*  *  *  * 

Tiie  FEEDBACK  is  separate 
coid  identified  and 
contains  some  HELPS. 


Like  the  previous  two 
call  signs j  the  STATEMENT 
for  task  organization 
call  signs  is  buried  in 
the  presentation. 

The  EXAMPLES  aren't  sep¬ 
arate  or  identified 
but  there  is  an  attempt 
to  restate  the  category 
characteristics . 

Again ,  there  is  no 
PRACTICE  REMEMBER  and 
the  PRACTICE  USING 
is  not  consistent. 
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The  FEEDBACK  is 
separate  and 
identified  and 
represents  the 
STATEMENT. 


FEEDBACK 


1. 

P2Q4 

6. 

(um) 

2. 

KULH 

7. 

(^Y82L) 

3. 

HE 

8. 

NOP 

4c 

(^Q2NY) 

9. 

BLUE  SMOKE 

5. 

HOPQ 

10.. 

KOU 

The  basic  pattern  for  task  organization  call  signs  is  lett 
number-1 etter-1 etter. 


Collective  Call  Signs 


Collective  call  signs  are  used  whenever  a  message  needs  to 
be  sent  to  several  commands  or  ships  instead  of  just  one. 

To  save  the  time  and  effort  that  would  be  needed  to  prepare 
a  message  addressed  to  each  command,  a  collective  call  sign 
is  used.  This  call  sign  is  one  that  automatically  addresses 
a  particular  group  of  commands.  For  example,  the  collective 
call  sign  NABT  means  **A1 1  ships  assigned  Greenland  patrol 
duty." 

Some  collective  call  signs  assigned  to  the  Navy  look  exactly 
like  international  call  signs— in  fact,  they  are  interna¬ 
tional  call  signs  that  have  been  designated  as  collective 
call  signs.  These  are  listed  in  ACP  113,  "Call  Sign  Book 
For  Ships."  In  section  4  of  ACP  113,  you  will  find  the  col¬ 
lective  call  signs  listed,  and  you  will  find  the  same  call 
signs  listed  in  the  section  of  ACP  113  that  lists  interna¬ 
tional  call  signs  for  Navy  ships.  Just  remember  that  a  col¬ 
lective  call  sign  is  an  international  call  sign  that  has 
been  given  a  special  meaning— that  of  addressing  a  specified 
group  of  commands. 

The  Navy  uses  fewer  collective  call  signs  than  other  types 
of  call  signs.  As  they  are  mostly  assigned  to  ships  operat¬ 
ing  in  a  certain  geographic  section  of  the  world,  not  many 
are  needed. 

Just  as  radiomen  must  learn  the  call  sign  for  their  ship, 
they  must  also  learn  the  collective  call  sign  for  the  area 
in  which  their  ship  is  operating.  This  comes  rather  quickly 
since  much  of  the  traffic  sent  to  a  ship  is  addressed  with  a 
collective  call  sign  and  the  radioman  normally  sees  it 
often. 


************ 


Circle  the  collective  call  signs  in  the  list  below  that 
could  be  used  by  the  Navy. 


NPE 

6. 

W2Z8 

P2ZN 

7. 

NOMV 

BUCKLE  ON 

8. 

KRLE 

NPDJ 

9. 

ND 

NA 

10. 

NTLQ 

***** 

*  * 

*  * 

Collective  call 
signs  were  not 
in  the  objective. 
Since  this  material 
is  irrelevant,  we 
will  not  rate  it 
for  adequacy. 


t 
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1.  NPE 

6.  W27.8 

2.  P2ZN 

7-& 

3.  BUCKLE  ON 

8.  KRLF. 

4.(nPDj) 

9.  ND 

5.  NA 

10.  (®) 

International  collective  call  signs  are  call  signs  that  have 
been  designated  for  the  specific  purpose  of  addressing  a 
group  of  commands  in  a  geographic  location.  They  look  ex¬ 
actly  like  international  call  signs  and  you  need  to  use 
ACP  113  in  order  to  tell  a  collective  call  from  an  indivi¬ 
dual  call. 


Net  call  signo 
were  not  in 
the  objective 
either ,  so  we 
will  not  rate 
this  section 
for  adequacy. 


Net  Call  Signs 

Net  call  signs  are  used  to  address  all  commands/stations 
that  are  using  a  particular  radio  net.  A  net  is  a  group  of 
stations  (not  necessarily  on  different  ship)  that  are  in 
direct  communication  over  ond  radio  network.  That  is, 
each  station  can  hear  the  calls  of  all  the  others  and 
each  station  can  call  any,  or  all,  of  the  others.  One 
example  of  a  net  call  sign  is  NQN.  This  means,  "All 
U.5.  Navy  Radio  Stations  guarding  the  ship-shore  high 
frequency  calling  series."  An  example  of  the  use  of  this 
call  sign  would  be  when  a  station  desired  to  communicate 
with  alj[  the  stations  at  the  same  time. 

Net  call  signs  are  also  used  on  voice  circuits.  An  example 
of  a  voice  net  call  sign  is  OVERWORK.  This  call  sign  means: 
"All  U.S.  Navy  ships  on  this  circuit."  Again,  a  net  call 
sign  would  be  used  when  the  calling  station  wishes  to  talk 
to  all  the  other  stations  at  once. 

These  two  examples  are  similar  to  other  types  of  call  signs. 
This  is  because  they  are  listed  along  with  other  types.  NQN 
is  found  in  the  listing  of  international  call  signs  and 
OVERWORK  is  listed  with  the  voice  call  signs. 

Net  call  signs  should  not  be  confused  with  collective  call 
signs.  Net  calls  always  specify  the  particular  circuit  over 
which  they  are  used.  A  collective  call  designates  certain 
commands  by  name  or  location  (geographic  area).  Net  call 
signs  are  either  international  call  signs  or  voice  call 
signs  so  designated  for  this  use. 
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*  *********** 


Check  (>4  the  two  items  below  that  describe  a  net  call  sign: 

a.  A  call  sign  represents  a  group  of  stations  in 
direct  communication  with  each  other  on  a  common 
channel . 

_  b.  A  call  sign  always  begins  with  the  letter  "M." 

_  c.  A  call  sign  may  be  composed  of  three  letters  or 

pronounceable  words. 

_  d.  A  call  sign  is  used  when  a  message  is  in  codress 

form. 

************ 


The  two  items  you  should  have  checked  which  describe  a  net 
call  sign  are: 


A  call  sign  represents  a  group  of  stations  in 
direct  communication  with  each  other  on  a  common 
channel . 


c.  A  call  sign  may  be  composed  of  three  letters  or 
pronounceable  words. 


Remember,  a  net  call  sign  may  be  an  international  or  a  voice 
call  sign  that  has  been  assigned  for  special  use.  Net  call 
signs  look  like  international  or  voice  call  signs,  and  their 
use  can  be  determined  only  by  consulting  the  proper  publica¬ 
tions. 


This  is  also 
irrelevant. 


h 


K 


37iis  polygraph 
is  irrelevant. 


Voice  Call  Signs 

Voice  call  signs,  as  the  name  implies,  are  used  over  voice 
radio  circuits.  These  consist  basically  of  words  or  com¬ 
binations  of  words  that  can  be  easily  pronounced.  For 
example,  the  Navy  ship  USS  Jason  is  assigned  the  voice  call 
sign  INKSTAND.  This  is  a  word  that  is  easily  pronounced, 
and  like  the  call  signs  discussed  earlier,  it  is  only 
assigned  to  one  command/ship. 

Just  as  there  are  collective  international  call  signs,  there 
are  also  collective  voice  call  signs.  There  is  no,  differ¬ 
ence  in  the  way  collective  voice  call  signs  and  those  for  a 
single  command  look.  However,  you  need  the  same  publica¬ 
tion,  JANAP  119(F),  "Joint  Voice  Call  Sign  Book"  in  order  to 
decode  and  encode  both  types. 

JANAP  119(F)  is  classified  Confidential  because  the  dis¬ 
closure  of  some  call  signs  and  the  commands  they  address  is 
a  violation  of  security. 


This  STATEMENT 
is  separated , 
but  not  identified. 


All  voice  call  signs  fall  into  four  types,  each  of  which 
listed  in  JANAP  119(F).  These  are: 

1.  Word  (or  words) 

2.  Word  (or  words)  plus  a  letter 

3.  Word  (or  words)  plus  a  digit 

4.  Digit  and  word  (or  words) 


is 


These  EXMIPLES 
are  identified , 
but  not  separated. 
There  is  a  little 
explanation  of 
how  letters  and 
digits  are  pro¬ 
nounced. 


Some  examples  are:  INKSTAND,  END  RUN  A  (ALPHA),  HAMLET  8 
(EIGHT),  and  6  (SIX)  WATERSHED.  (Note  that  letters  and 
digits  are  pronounced  as. words).  .Of  these  four  types  the 
last  one  (digit  and  word  or  words)  is  a  special  type  that 
is  reserved  for  special  use;  you  may  seldom  have  a  chance  to 
use  it. 


PRACTICE 


*********  *** 


Circle  the  voice  call  signs  in  the  list  below. 


This  PRACTICE 

A.  » 

BLUE  BOTTLE 

G. 

DSHD 

USING  is  separated 
and  identified } 
but  not  consistent. 

2. 

RUWSADF 

7. 

SEA  BOY  A 

3. 

USS  RONKARIS 

8. 

AIG  402 

4. 

NCCD 

9. 

SEX  APPEAL  1 

5. 

P7QR 

10. 

NA 

*  *  * 

*  *  *  * 

*  *  *  * 

i 

l 

1 

|  = 
I 

t 

s 

i 
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FEEDBACK 

i.  ^bluTbottle) 

6. 

DSWN 

2.  RUWSAOF 

7.< 

fSEA  BOY  A  (ALPHAl) 

3.  USS  RONKARIS 

8. 

AIG  402 

4.  UCCD 

9.1 

fsEX  APPEAL  1.'ToNe7 

5.  P7QR 

10. 

NA 

Yoice  call  signs  are  always  made  up  of  words,  digits,  or 
letters  that  are  pronounceable  (letters  are  always  pro¬ 
nounced  phonetically,  alpha,  bravo.  Charlie,  etc.). 


FINAL  PRACTICE 

************ 

You  should  now  be  able  to  identify  by  their  appearance, 
the  five  types  of  call  signs  listed  below.  Match  each  type 
of  call  sign  in  column  A  with  the  call  signs  in  column  B. 
Note  that  there  is  more  than  one  type  of  sane  call  signs  and 
that  some  items  in  colcnn  B  are  not  valid  call  signs. 


A 

B 

].  International 

A.  ROB INH00D  3 

(U.  S.  Navy  ships) 

B.  HG 

2.  International  (U.  S.  Navy 
shore  radio  stations) 

C.  NP0 

D.  DESRON  FOUR 

E.  S3BR 

3.  Task  Organization 

F.  HE AC 

4.  Indefinite 

6.  N7GL 

Again,  this  final 
pmctice  is  misleading, 
because  it  is  not 
consistent  with 
the  final  test. 

Remember,  the  student 
builds  expectations 
about  what  the  teat 
will  be  like  as  he 
does  the  practice. 

Here,  the  student 
expects  to  get  a 
similar  test  item 
on  the  final  test. 

That  is,  he  expects 
to  be  given  column 
A.  Therefore ,  he 
might  not  attempt 
to  memorize  the 
different  category 
names. 


5.  Voice  H.  HBRA 

6.  This  is  not  a 

valid  call  sign. 

_  K„  WAY  OUT  PAPA 

_  L.  COMNAVAIRPAC  TWO 

_  M.  NGA 

_  N.  NAPD 

0.  OVERLOAD 


I.  NPQL 

J.  NL 


************ 
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FEEDBACK 


5 

.  A- 

ROB INHOOD  3 

1 

_  I. 

NPQL 

4 

.  B- 

MG 

4 

.  J- 

NL 

2 

.  c* 

NPO 

5 

_  K. 

WAY  OUT  PAPA 

6_ 

.  D- 

DESRON  FOUR 

6 

_  L. 

COMNAVAIRPAC  TWO 

3 

_  E. 

S3BR 

2 

_  H* 

N5A 

1 

_  F. 

NEAC 

_ 1_ 

NAPD 

3 

.  G* 

N7GL 

5 

. 

_  0. 

OVERLOAD 

1 

H. 

NBRA 

This  feedback 
dees  restate 
the  category 
definitions. 

The  definition 
for  voice  calls, , 
however ,  is 
incomplete 


Two  of  these  are  not  valid  call  signs;  items  D  and  L  are 
plain  language  addresses.  Before  you  continue  with  the 
lesson  be  sure  that  you  understand  the  pattern  for  each  type 
of  call  sign: 

*  Voice  calls— pronounceable  word(s),  digit(s),  and 
letter(s) 

*  International  call  signs 

(ships)— four  letters  beginning  with  "H" 


(shore  radio  stations)— three  letters  beginning  with 


"N" 


(indefinite)— two  letters  beginning  with  *'N" 

(except  'JR) 

*  Task  organization  call  signs — ! etter-number-letter- 
letter  pattern 


Adequacy  Summary: 

None  of  the  STATEMENTS  are  separated  or  identified  carefully  enough. 

The  presentation  is  confusing,  and  too  wordy.  There  is  not  enough 
performance  orientation.  In  particular,  the  student  is  told  no  thirty 
about  how  to  study,  or  what  is  important  to  learn  for  the  final  test. 

The  EXAMPLES,  when  they  are  given  at  all,  do  not  illustrate  use  of 
the  category  characteristics.  None  of  the  examples  have  help  that 
explains  the  reasons  for  classification.  Further,  since  none  of  the 
examples  are  similar-  to  the  final  test  items,  we  can't  decide  if 
tMre  are  enough. 

The  PRACTICE  USING  is  all  inconsistent,  and  therefore  inadequate. 

The  FEEDBACK  is  separated  and  identified,  and  includes  help  or 
additional  explanation. 
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REVISED  VERSION  OF  EXAMPLE  ]. 


PUMPS 


In  your  job  aboard  ship,  you  will  need  to  know  how  to  operate.;, 
maintain,  and  repair  many  different  types  of  pumps.  Pumps  are 
important  parts  of  the  propulsion  and  fire  control  systems  on  a 
ship.  The  following  lessons  will  teach  you  the  part  names  of  the 
pumps  you  will  work  with  on  the  job.  You  must  know  the  part  names 
of  a  pump  in  order  to  operate,  maintain,  or  fix  it. 

The  Basic  Reciprocating  Pump 

The  first  pump  you  will  learn  about  is  the  basic  reciprocating 
pump.  Reciprocating  pumps  are  used  aboard  ship  to  move  water  through 
pipe  systems.  You  have  probably  used  a  basic  reciprocating  pump  to 
fill  a  basketball  or  bicycle  tire  with  air. 

The  parts  of  a  basic  reciprocating  pump  are  labeled  in  the 
diagram  below.  Study  the  name  of  each  part.  When  you  think  you 
have  memorized  the  part  names,  turn  the  page  to  the  PRACTICE 
section  and  try  to  label  the  diagram.  On  the  test  at  the  end  of 
this  lesson,  you  will  be  asked  to  label  a  diagram  like  this  one. 


PARTS  OF  A  BASIC  RECIPROCATING  PUMP; 


.CYLINDER 


1 


PISTON 


I - V 


a 


VALVE 


FT 


•HANDLE 


CONNECTING 

ROD 
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PRACTICE  SECTION 


J 


Label  the  diagram  below.  You  will  be  asked  to  label  a  diagram  exac tl> 
like  this  one  on  the  test  at  the  end  of  this  lesson. 

Check  yo»<r  answers  by  turning  back  to  the  previous  page.  If  you 
missed  any,  study  the  part  names  again.  When  you  are  ready,  you 
may  taka  the  lesson  test. 


REVISED  VERSION  OF  EXAMPLE  2. 


Sharpening  a  pocket  knife  of  a  sharpening  stone 

If  your  knife  is  not  sharp,  it  may  slip  and  damage  your  work 
or  cut  your  hand.  The  procedure  given  below  is  the  best 
way  to  sharpen  your  pocket  knife. 

You  will  be  tested  on  two  things:  First,  you  will 
have  to  remember  the  steps  for  sharpening.  On  the  test, 
you  will  have  to  write  the  steps  from  memory.  Second, 
you  will  sharpen  your  knife  while  your  instructor 
checks  you.  He  will  determine  whether  or  not  you 
follow  the  steps  correctly. 

PROCEDURE  FOR  SHARPENING 


Step  1.  Get  a  medium  or  fine  grade  sharpening 
stone. 

Step  2.  Spread  a  few  drops  of  oil  on  the 
surface  of  the  stone. 

Step  3.  Hold  the  handle  of  the  knife  in 
one  hand. 

Step  4.  Place  the  blade  across  the  stone  15°  > 

at  a  15°  angle.  ANGLE 

Step  5.  Press  the  blade  down  lightly  but  evenly 
with  the  fingers  of  the  other  hand. 

Step  6.  Stroke  the  blade  across  the  stone 

in  a  circular  motion  five  to  ten  times. 

Step  7.  Turn  the  blade  over,  so  that  the  other 
side  of  the  blade  is  at  a  15°  angle 
with  the  stone. 

Step  8.  Stroke  the  blade  across  the  stone 

in  a  circular  motion  five  to  ten  times. 

Step  9.  Rub  each  side  of  the  blade  on  a  soft 
block  of  canvas  or  leather. 


Test  Yourself: 

On  the  test,  you  will  have  to  write  the  steps  above 
from  memory.  Turn  this  page  over,  and  see  if  you  can 
write  each  step  from  memory  now.  Turn  back  to  this 
page  to  check  your  answers. 
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Example: 


Your  instructor  will  demonstrate  the  procedure  for  sharpening. 
He  will  show  you  each  of  the  steps  listed  above.  You  should  pay 
particular  attention  to  the  15°  angle,  and  how  the  knife  is  stroked 
on  the  stone. 


Practice: 

Practice  sharpening  your  knife.  Ask  your  instructor  to  check 
that  you  are  doing  each  step  correctly. 
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REVISED  VERSION  OF  EXAMPLE  3.  (This  example  centimes  over  Hie  next  C  pjaca 


CALL  SIGNS 


Introduction: 

Call  signs  are  used  by  all  radio  stations  to  identify  themselves. 

This  includes  commerical  stations,  like  KFMB;  Ham  radio  stations,  like 
WB6SQM;  police  stations,  like  "ONE  ADAM  TWELVE, "  and  many  others. 

The  Navy  also  uses  call  signs  to  identify  its  ship  and  shore 
radio  stations.  There  are  five  types  of  Navy  call  signs: 

1.  International  U.S.  Navy  Ship  Call  Signs. 

2.  International  U.S.  Navy  Shore  Call  Signs. 

3.  Indefinite  Call  Sighs. 

4.  Task  Organization  Call  Signs. 

5.  Voice  Call  Signs. 

On  the  job,  you  will  have  to  determine  the  type  of  any  call 
sign  you  might  be  given. 

There  are  two  things  you  will  be  tested  on  after  this  lesson: 

1.  You  will  have  to  write  from  memory  the  NAMES  and  DEFINITIONS 
of  each  of  the  five  types  of  call  signs.  You  should  memorize 
these  names  and  definitions. 

2,  We  will  give  you  a  list  of  call  signs  that  you  have  not  seen 
before.  For  each  call  sign  on  the  list,  you  will  have  to  decide 
which  type  it  is,  and  write  the  name  of  the  type  next  to  it. 

Some  signs  on  the  list  will  not  be  valid  Navy  call  signs. 

For  those,  you  will  just  write  "invalid." 


How  to  study  this  lesson: 

On  page  2,  you  will  find  the  NAMES  and  DEFINITIONS  of  each  of  the 
five  types  of  Navy  call  signs.  You  should  memorize  these. 

On  page  3,  you  can  practice  remembering  the  names  and  definitions 
just  like  you  will  get  on  the  test.  The  answers  are  on  page  2. 

On  page  4,  we  will  give  a  list  of  example  call  signs,  and  show  you 
how  to  decide  which  of  the  five  types  they  are. 

On  page  5>  we  will  give  you  a  list  of  call  signs  just  like  you 
will  get  on  the  test.  For  each  of  these,  you  can  practice  deciding 
which  type  of  call  sign  it  is.  The  answers  are  on  page  6. 

When  you  think  you  can  answer  all  the  test  questions,  give  this  booklet 
back.  Then  you  will  be  given  the  test.  Your  test  will  be  scored 
right  after  you  finish  so  you  will  know  how  you  did. 

Remember,  don't  take  the  test  until  you  are  sure  you'll  get  it  right. 
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Five  types  of  CALL  SIGNS,  NAMFS  and  DEFINITIONS 


To  Study:  mWRIZE  the  NAMES  and  DEFINITIONS  given  below ,  You  don’t 
have  to  memorize  the  explanations  in  italics. 


NAME 


DEFINITION 


Explanation 


1. 

INTERNATIONAL  U.S. 

NAVY  SHIP 

STARTS  with  %" 
has  THREE  more  LETTERS. 

Remember: 

"N"  +  3  more  letters. 

2. 

INTERNATIONAL  U.S. 

NAVY  SHORE 

STARTS  with  "N," 
has  TWO  more  LETTERS. 

Remember: 

"N"  +  2  more  letters. 

3. 

INDEFINITE 

STARTS  with  %" 

has  CNE  more  LETTER  (NOT  "R"). 

Remember: 

"N"  4  any  LETTER  except  "R." 

Remember : 

"NR"  is  Not  Really  a  call  sign. 

Every  ship  has  its  ox,m 
international  call  sign 
so  it  can  identify  itself 
both  in  port  and  at  sea. 

Shore  radio  stations  also 
have  international  call 
signs  to  identify  them¬ 
selves. 


Indefinite  call  signs  are 
used  when  the  identity  of 
a  station  must  be  kepi 
secret.  These  are  used 
on  coded  ("codress") 
messages. 

Task  organization  call 
signs  are  used  to  refer 
to  groups  of  ships  during 
tactical  exercises.  V/hen 
the  operation  is  over, 
ships  may  change  groups, 
but  the  task  organization 
call  sign  slays  with  the 

if!  • 


Task  Organization  call  signs  are 
listed  in  ACP  122(b)  and 
ACP  112  U.S.  SUPP(A)-1.  Both  of 
these  are  CONFIDENTIAL. 


Voice  call  signs  are  used 
on  voice  radio  circuits . 
they  are  words  that  can 
be  pronounced  easily, 
numbers  dr  letters  are 
also  pronounced,  like 
"ALPHA"  for  "A,"  or 
"EIGHT"  for  "8. " 

Voice  call  signs  are  listed  in 
JAMP  119(f)  which  is  CONFIDENTIAL. 

There  are  other  types  of  call  signs,  but  you  need  books  to  look  them  up.  You 
can't  tell  '’’em  apart  any  other  way.  will  learn  about  them  in  a  later  lesson. 


PRACTICE: 


On  the  test,  you  will  have  to  write  the  names  and  definitions  of 
the  five  types  of  call  signs.  In  the  space  below,  see  if  you  can 
write  them  now.  You  have  to  be  able  to  do  this  without  looking  at 
page  2. 


Mi _  DEFINITION 


i 


1 


-  -■  i 


*  i 


P 

\ 

1 

1 

V 

i 

1 

1 

i 

1 

i 

I 

Turn  back  to  page  2  to  check  your  answers. 
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EXAMPLES 


On  the  test,  we  will  give  you  a  list  of  call  signs.  You  will  have  to 
decide  which  type  each  one  is,  and  write  down  the  type.  We  have  done  some 
samples  below.  Study  these  so  you  understand  how  to  decide  on  the  type. 


Call  Siqn 

TYPE 

EXPLANATION 

NDCP 

Ivtfer 

O.b.K^Y  &hip 

"N"  +  3  move  letters. 

NR 

jAjtf#  Ul2> 

NR  is  NEVER  USED  as  a 

Navy  call  sign. 

Remember,  "Not  Really" 

NPX 

u.s>.  yy  shon? 

"N"  +  2  move  letters. 

PORK  CHOP  THREE 

VOic£" 

WORDS  plus  a  NUMBER 

USS  CONSTELLATION 

i  NVA  u 

"USS"  is  an  abbreviation. 

NUWEPCEN 

lMV  A  U& 

This  is  an  abbreviation. 

Even  though  it  starts  with 
"N, "  it  has  too  many  letters. 

NT4 

|NV7H-it> 

This  starts  with  "N",  but  it 
doesn't  have  2  more  letters. 

Also ,  it  isn't  letter-number- 
letier-letter. 

NTP 

\*T&**WAri  CAUC¬ 
US  .  M*vy  skoAe 

"N"  +  two  move  letters. 

CRUDESRON  FIVE 

(Ml/ALlb 

" CRUDESRON "  is  an  abbreviation. 

N3RT 

Task 

letter-number-letter-letter. 

NB 

iUbETFi/V/nS: 

"N"  +  one  move  letter. 

SEVEN  ICEBERG 

1/dicS” 

NUMBER  plus  WORD.  ( 

R9FH 

Tft5K 
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letter-number-letter-letter  . 

i 

O 


PKmoi ICE 


On  the  test,  you  will  be  given  a  list  of  call  signs,  and  you  will  have 
to  write  which  type  each  one  is.  For  practice,  we  have  given  a  list  of 
call  signs  below.  Next  to  each  one,  write  its  type.  If  it  is  not  a  valid 
Navy  call  sign,  write  "invalid."  Answers  are  on  the  next  page. 

Call  Sign _ JYPE _ 

R0BINH00D  THREE 


NG 


NPQ 


DESRON  FOUR 


S3BR 


NEAC 


N7GL 


NBRA 


NPQL 


NL 


WAY  OUT  PAPA 


COMNAVAIRPAC  TWO 


NR 


NGA 


NAPD 


OVERLOAD 


NMR4 
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ANSWERS 


Call  Sign 

TYPE 

Explanation 

ROB INHOOD  THREE 

Vfoi 

WORD  plus  UflHBER. 

NG 

|Wt>E"FiKi  \TC 

"N"  +  any  letter  except  "R" 

NPQ 

WferjaaTiowcu..  O.S. 

AWy  $Hdp.E 

"If"  +  2  move  letters. 

DESRON  FOUR 

DESRON  is  an  abbi'evuiticm. 

S3BR 

letter-numbei'-letter-lctter 

NEAC 

5h)'p 

"N"  -/•  S  more  letters. 

N7GL 

Task  6&&6MZ.CCJ7MJ 

letter-number-letter- letter . 

mn  nil 

fjcury  Ship 

"N"  +  2  more  letters. 

NPQL 

UTe»*/UfcTr*AMj  U‘S, 

"If"  -f  3  more  letters. 

NL 

IvabeFiMiTe 

"N"  +  any  letter  except  "R. " 

WAY  OUT  PAPA 

\foicG. 

WORDS  +  LETTER  ("P"=PAPA) 

COMNAVAIRPAC  TWO 

COMNAVAIRPAC  is  abbreviation 

NR 

"NR"  is  NEVER  used  as  call  sign 

NGA 

l^rK>eOTu>u<\L.  (x$. 

/Jav'y  Sh&fte 

"N"  +  2  more  letters. 

NAPD 

■  \teriOAtj*e»MCv.u.  U  •  $  * 

A;^y  Ship 

"N"  -r  3  more  letters. 

OVERLOAD 

Vt>\  C€ 

WORD 

NMR4 

|/\yahc\ 

Starts  with  "N, "  but  doesn’t 
have  3  letters.  Also  not 
letter-number- letter- letter 

If  you  didn't  get  all  these  right,  you  should  review  the  previous  pages . 
You  may  ask  to  take  the  test  whenever  you  j eel  ready. 
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CONCLUSION 


This  concludes  Volume  II  of  the  IQI,  the  User's  Manual.  This 
volume  is  intended  to  provide  additional  explanation  and  examples 
of  the  application  of  the  procedures  in  Volume  IV. 

Two  points  made  in  this  volume  must  be  reemphasized:  First, 
the  IQI  is  not  a  set  of  rigid  procedures  that  must  be  meticulously 
followed  in  all  situations.  Instead,  the  IQI  should  be  viewed  as 
a  set  of  guidelines  designed  to  help  the  user  ask  appropriate 
questions  about  what  is  to  be  taught,  and  how  it  should  be  taught. 

Second,  the  most  important  thing  to  keep  in  mind  while  using  the 
IQI  is  "REMEMBER  THE  JOB."  All  too  often,  users  of  procedures  like 
this  become  lost  in  the  procedural  details,  and  forget  about  what 
they  are  trying  to  accomplish. 


For  More  Information: 

Questions  or  comments  on  the  IQI  should  be  addressed  to  the 
authors  at: 

Navy  Personnel  Research  and  Development  Center 
San  Diego,  CA  92152 

Autovon:  933-7121 

7140 
7194 


Commercial:  (714)  225-7121 
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